Jul 28 19:02:38.415211 ip-10-0-138-64 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 19:02:38.830496 ip-10-0-138-64 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:02:38.830496 ip-10-0-138-64 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 19:02:38.830496 ip-10-0-138-64 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:02:38.830496 ip-10-0-138-64 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 19:02:38.830496 ip-10-0-138-64 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:02:38.831727 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.831637    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 19:02:38.836732 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836712    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:38.836732 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836729    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:38.836732 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836734    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:38.836732 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836738    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836743    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836748    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836751    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836755    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836758    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836762    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836766    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836769    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836773    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836776    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836780    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836784    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836788    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836796    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836800    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836804    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836808    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836812    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836816    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:38.837007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836819    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836823    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836827    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836831    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836838    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836844    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836849    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836854    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836859    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836863    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836868    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836874    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836878    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836882    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836887    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836892    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836896    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836900    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836906    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:38.837887 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836913    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836936    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836941    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836945    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836949    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836953    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836957    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836961    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836965    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836969    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836974    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836979    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836983    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836987    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836991    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.836996    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837000    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837004    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837007    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837011    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:38.838712 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837016    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837020    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837025    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837029    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837033    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837037    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837042    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837047    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837051    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837056    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837060    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837064    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837069    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837073    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837079    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837085    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837090    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837094    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837098    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837102    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:38.839569 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837106    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837110    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837114    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837117    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837122    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837126    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837133    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837138    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837143    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837147    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837151    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837155    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837159    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837163    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837167    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837172    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.837176    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838344    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838354    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:38.840434 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838359    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838364    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838368    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838373    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838377    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838381    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838385    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838390    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838394    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838400    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838404    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838408    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838412    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838417    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838421    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838425    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838429    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838433    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838437    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838443    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:38.841267 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838449    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838453    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838457    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838462    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838466    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838470    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838476    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838481    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838485    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838489    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838494    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838498    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838502    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838506    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838511    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838515    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838519    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838523    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838527    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838532    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:38.841852 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838536    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838540    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838546    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838550    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838554    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838558    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838563    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838568    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838572    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838576    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838580    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838584    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838588    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838592    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838597    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838605    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838611    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838616    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838621    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:38.842361 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838625    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838629    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838633    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838637    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838641    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838646    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838650    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838656    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838660    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838665    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838669    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838672    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838677    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838681    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838685    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838689    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838694    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838698    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838703    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838707    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:38.842943 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838711    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838714    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838719    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838722    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838726    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838730    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838735    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838739    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838743    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838748    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838752    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838757    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838763    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838769    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838773    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838778    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838782    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.838787    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839432    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839440    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:38.843597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839446    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839451    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839456    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839460    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839465    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839469    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839476    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839482    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839488    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839493    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839497    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839501    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839506    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839510    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839514    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839518    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839523    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839526    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839531    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839537    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:38.844176 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839543    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839548    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839552    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839558    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839562    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839567    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839571    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839576    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839580    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839584    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839588    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839592    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839596    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839600    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839604    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839608    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839613    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839617    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839621    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839625    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:38.844725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839629    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839634    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839638    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839642    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839646    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839650    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839655    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839659    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839663    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839667    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839671    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839675    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839679    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839683    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839688    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839693    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839697    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839702    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839706    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:38.845277 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839710    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839714    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839719    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839723    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839727    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839731    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839735    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839739    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839743    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839747    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839751    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839757    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839763    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839767    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839771    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839776    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839781    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839786    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839791    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839796    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:38.845735 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839800    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839803    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839807    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839811    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839815    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839820    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839824    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839828    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839832    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839836    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839840    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839844    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839848    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839853    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839857    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839862    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839866    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.839870    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.839993    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840005    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 19:02:38.846332 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840014    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840021    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840027    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840032    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840038    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840045    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840050    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840055    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840062    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840068    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840073    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840077    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840082    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840087    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840091    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840096    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840103    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840108    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840112    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840117    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840123    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840129    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840134    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 19:02:38.846810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840139    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840144    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840149    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840154    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840159    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840164    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840168    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840175    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840179    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840184    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840188    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840193    2563 flags.go:64] FLAG: --enable-server="true"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840197    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840204    2563 flags.go:64] FLAG: --event-burst="100"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840209    2563 flags.go:64] FLAG: --event-qps="50"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840213    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840218    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840223    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840230    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840235    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840240    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840244    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840249    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840254    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840259    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 19:02:38.847387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840263    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840268    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840272    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840276    2563 flags.go:64] FLAG: --feature-gates=""
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840283    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840288    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840293    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840297    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840302    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840307    2563 flags.go:64] FLAG: --help="false"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840315    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-138-64.ec2.internal"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840320    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840326    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840330    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840336    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840341    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840346    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840350    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840354    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840358    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840363    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840368    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840373    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840377    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 19:02:38.848130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840382    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840387    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840393    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840397    2563 flags.go:64] FLAG: --lock-file=""
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840402    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840407    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840411    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840420    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840425    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840429    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840433    2563 flags.go:64] FLAG: --logging-format="text"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840438    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840443    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840448    2563 flags.go:64] FLAG: --manifest-url=""
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840452    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840459    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840464    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840470    2563 flags.go:64] FLAG: --max-pods="110"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840479    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840484    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840489    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840493    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840498    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840502    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840508    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 19:02:38.849155 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840519    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840524    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840528    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840533    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840537    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840546    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840550    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840556    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840560    2563 flags.go:64] FLAG: --port="10250"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840565    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840570    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d4057469b2634d7e"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840576    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840580    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840585    2563 flags.go:64] FLAG: --register-node="true"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840589    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840596    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840600    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840604    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840609    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840615    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840620    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840625    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840649    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840656    2563 flags.go:64] FLAG: --runonce="false"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840661    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 19:02:38.850249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840666    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840673    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840678    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840682    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840687    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840692    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840699    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840703    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840708    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840713    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840717    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840722    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840727    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840731    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840740    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840744    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840749    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840756    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840762    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840766    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840771    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840775    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840780    2563 flags.go:64] FLAG: --v="2"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840786    2563 flags.go:64] FLAG: --version="false"
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840792    2563 flags.go:64] FLAG: --vmodule=""
Jul 28 19:02:38.851120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840798    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.840803    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.840998    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841007    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841013    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841020    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841027    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841031    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841038    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841044    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841048    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841053    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841057    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841064    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841068    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841073    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841077    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841081    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841085    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841089    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:38.851946 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841093    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841097    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841101    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841105    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841110    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841114    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841119    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841123    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841127    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841131    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841135    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841140    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841144    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841148    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841152    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841156    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841160    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841164    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841169    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841174    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:38.852801 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841179    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841183    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841187    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841191    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841198    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841204    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841209    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841213    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841216    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841220    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841225    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841229    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841233    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841237    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841242    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841246    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841250    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841255    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841267    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:38.853442 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841272    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841276    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841280    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841284    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841288    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841292    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841296    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841300    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841305    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841308    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841315    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841320    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841324    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841330    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841335    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841339    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841343    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841348    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841354    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:38.854221 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841358    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841363    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841367    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841371    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841375    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841379    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841383    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841387    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841392    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841396    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841400    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841404    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841408    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841412    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841417    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841421    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841425    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841429    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841433    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841437    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:38.855130 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841441    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841445    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.841449    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.842400    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.850610    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.850630    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.851489    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.851504    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851596    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851604    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851609    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851615    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851619    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851623    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851628    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:38.855659 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851632    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851636    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851640    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851645    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851649    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851654    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851658    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851662    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851666    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851670    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851674    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851678    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851682    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851686    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851691    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851695    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851699    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851703    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851707    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851711    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851715    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:38.856184 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851719    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851731    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851736    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851740    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851744    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851748    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851752    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851756    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851760    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851767    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851773    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851778    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851782    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851787    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851791    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851795    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851800    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851804    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851808    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:38.856725 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851812    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851816    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851821    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851825    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851829    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851833    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851837    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851841    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851845    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851849    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851853    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851858    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851862    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851866    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851870    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851881    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851891    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851896    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851900    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851904    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:38.857284 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851909    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851932    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851939    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851943    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851947    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851953    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851958    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851963    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851967    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851971    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851975    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851980    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851984    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851988    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851993    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.851997    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852001    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852005    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:38.857817 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852009    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852013    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852017    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852021    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852025    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852029    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852033    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852037    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852041    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852045    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852057    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852062    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852066    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852070    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.852078    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:02:38.858266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852297    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852306    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852311    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852316    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852320    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852324    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852331    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852337    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852341    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852345    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852349    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852354    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852358    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852362    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852366    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852370    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852374    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852378    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852382    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:38.858653 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852386    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852390    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852394    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852398    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852403    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852407    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852411    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852415    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852419    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852431    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852436    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852440    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852444    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852448    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852452    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852456    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852460    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852464    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852469    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852473    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:38.859154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852477    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852481    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852485    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852490    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852494    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852498    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852502    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852506    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852510    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852515    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852519    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852523    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852527    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852531    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852534    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852539    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852543    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852547    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852551    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852555    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:38.859871 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852559    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852563    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852575    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852579    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852584    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852588    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852592    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852596    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852601    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852607    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852611    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852616    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852620    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852624    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852628    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852632    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852636    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852642    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:38.860562 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852647    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852652    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852657    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852661    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852665    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852669    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852673    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852677    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852682    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852686    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852690    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852694    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852698    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852702    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852706    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852711    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852715    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852726    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852731    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852735    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:38.861275 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852739    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:38.861805 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:38.852743    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:38.861805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.852751    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:02:38.861805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.852954    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 19:02:38.861805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.855631    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 19:02:38.861805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.857050    2563 server.go:1019] "Starting client certificate rotation"
Jul 28 19:02:38.861805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.857165    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:02:38.861805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.857208    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:02:38.861805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.860645    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:02:38.864360 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.864344    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:02:38.874067 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.874050    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 28 19:02:38.876051 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.876033    2563 log.go:25] "Validated CRI v1 image API"
Jul 28 19:02:38.876869 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.876848    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 19:02:38.879492 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.879466    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8b85380a-3143-47c7-b8d5-ee649f4b55f4:/dev/nvme0n1p4 eea646b1-8476-4ec8-bce0-e7e697cee14e:/dev/nvme0n1p3]
Jul 28 19:02:38.879540 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.879493    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 19:02:38.885976 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.885851    2563 manager.go:216] Machine: {Timestamp:2026-07-28 19:02:38.885322726 +0000 UTC m=+0.346930423 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099157 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e614bcb7d9fd5d62fd9a96bbe0fa5 SystemUUID:ec2e614b-cb7d-9fd5-d62f-d9a96bbe0fa5 BootID:a5a1f62d-680f-445b-9f0d-c89141cd0836 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:21:8b:bd:b0:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:21:8b:bd:b0:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:d7:ab:8d:bf:44 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 19:02:38.885976 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.885973    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 19:02:38.886117 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.886053    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 19:02:38.887315 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.887299    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:02:38.889288 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.889250    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 19:02:38.889441 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.889292    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-64.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 19:02:38.890160 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.890150    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 19:02:38.890190 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.890162    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 19:02:38.890190 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.890169    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:02:38.890190 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.890179    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:02:38.890930 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.890908    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 19:02:38.892725 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.892715    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:02:38.892840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.892830    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 19:02:38.892880 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.892862    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 19:02:38.892880 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.892874    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 19:02:38.892983 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.892892    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 19:02:38.892983 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.892902    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 28 19:02:38.892983 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.892912    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 19:02:38.893348 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.893332    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:02:38.893407 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.893357    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:02:38.897423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.897398    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 19:02:38.899501 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.899485    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 19:02:38.899562 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.899516    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 19:02:38.900847 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900834    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 19:02:38.900901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900852    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 19:02:38.900901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900859    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 19:02:38.900901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900866    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 19:02:38.900901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900876    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 19:02:38.900901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900882    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 19:02:38.900901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900891    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 19:02:38.900901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900897    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 19:02:38.901102 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900905    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 19:02:38.901102 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900912    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 19:02:38.901102 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900936    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 19:02:38.901102 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.900943    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 19:02:38.901859 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.901849    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 19:02:38.901859 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.901858    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 19:02:38.903069 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.903047    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 19:02:38.903131 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.903050    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-64.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 19:02:38.905165 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.905153    2563 server.go:1268] "Started kubelet"
Jul 28 19:02:38.905350 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.905279    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 19:02:38.905350 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.905254    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 19:02:38.905465 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.905371    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 19:02:38.906042 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.906023    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-64.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 19:02:38.906111 ip-10-0-138-64 systemd[1]: Started Kubernetes Kubelet.
Jul 28 19:02:38.907574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.907548    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 19:02:38.908030 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.908012    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 19:02:38.909418 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.909396    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kjqdp"
Jul 28 19:02:38.912628 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.912609    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 19:02:38.912972 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.911790    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-64.ec2.internal.18c689629b59ab15  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-64.ec2.internal,UID:ip-10-0-138-64.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-64.ec2.internal,},FirstTimestamp:2026-07-28 19:02:38.905125653 +0000 UTC m=+0.366733347,LastTimestamp:2026-07-28 19:02:38.905125653 +0000 UTC m=+0.366733347,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-64.ec2.internal,}"
Jul 28 19:02:38.916459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.916442    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 19:02:38.916459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.916452    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 19:02:38.916573 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.916520    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:38.917035 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917018    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 19:02:38.917035 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917032    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 19:02:38.917153 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917045    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 19:02:38.917194 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917168    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 19:02:38.917194 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917177    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 19:02:38.917303 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917205    2563 factory.go:55] Registering systemd factory
Jul 28 19:02:38.917303 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917242    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 28 19:02:38.917434 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917418    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kjqdp"
Jul 28 19:02:38.917827 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917777    2563 factory.go:153] Registering CRI-O factory
Jul 28 19:02:38.917827 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917795    2563 factory.go:223] Registration of the crio container factory successfully
Jul 28 19:02:38.917961 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917861    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 19:02:38.917961 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917885    2563 factory.go:103] Registering Raw factory
Jul 28 19:02:38.917961 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.917901    2563 manager.go:1196] Started watching for new ooms in manager
Jul 28 19:02:38.918480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.918466    2563 manager.go:349] Starting recovery of all containers
Jul 28 19:02:38.919272 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.919247    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 19:02:38.919362 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.919336    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-64.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 19:02:38.926327 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.926294    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 19:02:38.929996 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.929859    2563 manager.go:354] Recovery completed
Jul 28 19:02:38.931293 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.931267    2563 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 19:02:38.934362 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.934348    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:38.936090 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.936074    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:38.936158 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.936106    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:38.936158 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.936118    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:38.936634 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.936620    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 19:02:38.936634 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.936632    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 19:02:38.936724 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.936647    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:02:38.936724 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.936696    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:38.937380 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.937334    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:38.937380 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.937361    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:38.937380 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.937374    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:38.938520 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.938510    2563 policy_none.go:49] "None policy: Start"
Jul 28 19:02:38.938557 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.938525    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 19:02:38.938557 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.938536    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 19:02:38.939552 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.939541    2563 policy_none.go:47] "Start"
Jul 28 19:02:38.979501 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.979484    2563 manager.go:337] "Starting Device Plugin manager"
Jul 28 19:02:38.979610 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.979516    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 19:02:38.979610 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.979526    2563 server.go:87] "Starting device plugin registration server"
Jul 28 19:02:38.979767 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.979754    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 19:02:38.979802 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.979773    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 19:02:38.979873 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.979859    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 19:02:38.979974 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.979957    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 19:02:38.979974 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:38.979969    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 19:02:38.980451 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.980434    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 19:02:38.980510 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:38.980479    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.080142 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.080115    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:39.081299 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.081253    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:39.081361 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.081301    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:39.081361 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.081313    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:39.081361 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.081341    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.087327 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.087312    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.087382 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.087334    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-64.ec2.internal\": node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.098737 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.098711    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.137255 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.137221    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 19:02:39.137399 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.137271    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 19:02:39.137399 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.137291    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 19:02:39.137399 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.137326    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 19:02:39.139519 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.139499    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:39.199616 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.199579    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.238057 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.238015    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal"]
Jul 28 19:02:39.238178 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.238104    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:39.239084 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.239068    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:39.239171 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.239103    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:39.239171 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.239116    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:39.240304 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.240288    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:39.240419 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.240403    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.240462 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.240442    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:39.241026 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.241008    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:39.241132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.241032    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:39.241132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.241046    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:39.241132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.241067    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:39.241132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.241090    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:39.241132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.241104    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:39.242257 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.242242    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.242336 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.242273    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:39.242887 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.242872    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:39.242982 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.242900    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:39.242982 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.242913    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:39.264486 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.264465    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-64.ec2.internal\" not found" node="ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.269068 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.269052    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-64.ec2.internal\" not found" node="ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.300398 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.300365    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.319435 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.319417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/20fec549702e81f5927fb23efca53227-config\") pod \"kube-apiserver-proxy-ip-10-0-138-64.ec2.internal\" (UID: \"20fec549702e81f5927fb23efca53227\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.319537 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.319442    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/79fa5413215af614dae2a46132878cd3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal\" (UID: \"79fa5413215af614dae2a46132878cd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.319537 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.319461    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fa5413215af614dae2a46132878cd3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal\" (UID: \"79fa5413215af614dae2a46132878cd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.401286 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.401182    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.420566 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.420534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/20fec549702e81f5927fb23efca53227-config\") pod \"kube-apiserver-proxy-ip-10-0-138-64.ec2.internal\" (UID: \"20fec549702e81f5927fb23efca53227\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.420686 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.420571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/79fa5413215af614dae2a46132878cd3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal\" (UID: \"79fa5413215af614dae2a46132878cd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.420686 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.420595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fa5413215af614dae2a46132878cd3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal\" (UID: \"79fa5413215af614dae2a46132878cd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.420686 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.420642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/79fa5413215af614dae2a46132878cd3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal\" (UID: \"79fa5413215af614dae2a46132878cd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.420686 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.420642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fa5413215af614dae2a46132878cd3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal\" (UID: \"79fa5413215af614dae2a46132878cd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.420686 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.420642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/20fec549702e81f5927fb23efca53227-config\") pod \"kube-apiserver-proxy-ip-10-0-138-64.ec2.internal\" (UID: \"20fec549702e81f5927fb23efca53227\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.501737 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.501703    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.567305 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.567270    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.572138 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.572115    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:39.602407 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.602367    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.703578 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.703548    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.808589 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.808555    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.856678 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.856646    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 19:02:39.857178 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.856850    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:02:39.909088 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:39.909063    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:39.922292 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.922257    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 18:57:39 +0000 UTC" deadline="2028-05-11 15:11:39.49749708 +0000 UTC"
Jul 28 19:02:39.922292 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.922285    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15668h8m59.575215661s"
Jul 28 19:02:39.928067 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.928046    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:39.987325 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:39.987257    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:40.009671 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:40.009645    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:40.110307 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:40.110272    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:40.161317 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:40.161282    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79fa5413215af614dae2a46132878cd3.slice/crio-32846caa5522f50cfe349020d6fefddc8e5c378daed88608846795fc30834970 WatchSource:0}: Error finding container 32846caa5522f50cfe349020d6fefddc8e5c378daed88608846795fc30834970: Status 404 returned error can't find the container with id 32846caa5522f50cfe349020d6fefddc8e5c378daed88608846795fc30834970
Jul 28 19:02:40.163182 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:40.163152    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20fec549702e81f5927fb23efca53227.slice/crio-0736e637e2cc99bb467013b2be9e3e315cd04cc48a92d223ac815c2e118167bf WatchSource:0}: Error finding container 0736e637e2cc99bb467013b2be9e3e315cd04cc48a92d223ac815c2e118167bf: Status 404 returned error can't find the container with id 0736e637e2cc99bb467013b2be9e3e315cd04cc48a92d223ac815c2e118167bf
Jul 28 19:02:40.167936 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.167915    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:02:40.210972 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:40.210914    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:40.317584 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:40.317487    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:40.417879 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:40.417833    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-64.ec2.internal\" not found"
Jul 28 19:02:40.438643 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.438615    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:40.517817 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.517776    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:40.529941 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.529899    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:02:40.530958 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.530942    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal"
Jul 28 19:02:40.538425 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.538398    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:02:40.756548 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.756467    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:40.893909 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.893879    2563 apiserver.go:52] "Watching apiserver"
Jul 28 19:02:40.901527 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.901496    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 19:02:40.904033 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.903999    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-29nx8","openshift-network-operator/iptables-alerter-f67ln","openshift-ovn-kubernetes/ovnkube-node-62f22","kube-system/global-pull-secret-syncer-wl5w9","kube-system/konnectivity-agent-jnj67","kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl","openshift-cluster-node-tuning-operator/tuned-x7r6w","openshift-dns/node-resolver-ckcmv","openshift-image-registry/node-ca-9k5kh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal","openshift-multus/multus-additional-cni-plugins-8lhnl","openshift-multus/multus-lsrbm","openshift-multus/network-metrics-daemon-9xt7v"]
Jul 28 19:02:40.905754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.905731    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:40.907374 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.907353    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.908724 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.908705    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.908905 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.908884    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 19:02:40.909017 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.908963    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 19:02:40.909167 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.909148    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jkfjf\""
Jul 28 19:02:40.909295 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.909278    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 19:02:40.910102 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.910079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:40.910208 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:40.910151    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:02:40.910208 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.910091    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 19:02:40.910484 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.910467    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vl5q9\""
Jul 28 19:02:40.910767 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.910746    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 19:02:40.910881 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.910771    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 19:02:40.911025 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.911008    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 19:02:40.912304 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.911590    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q2rxx\""
Jul 28 19:02:40.912304 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.911618    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:02:40.912304 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.911842    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 19:02:40.912304 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.911867    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 19:02:40.912304 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.912241    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 19:02:40.913116 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.913088    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 19:02:40.917397 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.913555    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 19:02:40.917397 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.915324    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:40.917397 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:40.915439    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:02:40.917397 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.916241    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:40.917397 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.916740    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 19:02:40.917872 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.917845    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:40.917951 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:40.917927    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:02:40.918434 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.918410    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 19:02:40.918633 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.918614    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6sdsm\""
Jul 28 19:02:40.918762 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.918745    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 19:02:40.918826 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.918764    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 19:02:40.919740 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.919720    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 19:02:40.919956 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.919918    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-glwzt\""
Jul 28 19:02:40.920028 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.919970    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 19:02:40.920559 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.920541    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:40.922321 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.922303    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:40.923852 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.923836    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:40.924256 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.924234    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fkv95\""
Jul 28 19:02:40.924648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.924627    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 19:02:40.924732 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.924655    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:02:40.924800 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.924656    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 19:02:40.925173 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.925017    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 19:02:40.925173 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.925111    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 19:02:40.925342 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.925198    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 19:02:40.925342 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.925208    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nrqz4\""
Jul 28 19:02:40.925560 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.925542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:40.926764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.926743    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 19:02:40.928578 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.928578 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928547    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:02:40.928578 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928568    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7c6fda6c-c699-4ce6-a78a-209994e2db79-kubelet-config\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:40.928770 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed3429b6-f5f2-48ab-a368-28543e3860ad-hosts-file\") pod \"node-resolver-ckcmv\" (UID: \"ed3429b6-f5f2-48ab-a368-28543e3860ad\") " pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:40.928770 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928622    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 19:02:40.928770 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02b6ba59-b83b-4614-a80e-4802ed5d66d5-ovn-node-metrics-cert\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.928770 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7c6fda6c-c699-4ce6-a78a-209994e2db79-dbus\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:40.928770 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2w9f\" (UniqueName: \"kubernetes.io/projected/ed3429b6-f5f2-48ab-a368-28543e3860ad-kube-api-access-t2w9f\") pod \"node-resolver-ckcmv\" (UID: \"ed3429b6-f5f2-48ab-a368-28543e3860ad\") " pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:40.928770 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-etc-kubernetes\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.928770 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928781    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-socket-dir-parent\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8dc3ae5c-574b-40bc-a2e5-24e47ffec047-iptables-alerter-script\") pod \"iptables-alerter-f67ln\" (UID: \"8dc3ae5c-574b-40bc-a2e5-24e47ffec047\") " pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-registration-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-kubelet\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928903    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-node-log\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928940    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-p7xcd\""
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.928962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-run-netns\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02b6ba59-b83b-4614-a80e-4802ed5d66d5-ovnkube-script-lib\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929041    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g256\" (UniqueName: \"kubernetes.io/projected/88ae7e56-364a-4f77-862a-2065daefbcfc-kube-api-access-5g256\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-cni-dir\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n2zk\" (UniqueName: \"kubernetes.io/projected/bcb8000e-6bba-4dbc-a735-1022c0c72320-kube-api-access-9n2zk\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.929193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-socket-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-systemd-units\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-log-socket\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-cni-netd\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hp4z\" (UniqueName: \"kubernetes.io/projected/02b6ba59-b83b-4614-a80e-4802ed5d66d5-kube-api-access-4hp4z\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-var-lib-cni-bin\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929415    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-slash\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-run-k8s-cni-cncf-io\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-run-netns\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929555    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-system-cni-dir\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-run-multus-certs\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-device-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8wrg\" (UniqueName: \"kubernetes.io/projected/0dff55ad-8180-488d-9996-3a9e46a31c44-kube-api-access-w8wrg\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-run-systemd\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929660    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cpzxr\""
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-etc-openvswitch\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.929898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-cnibin\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-var-lib-kubelet\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-var-lib-openvswitch\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-run-ovn\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbwps\" (UniqueName: \"kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps\") pod \"network-check-target-29nx8\" (UID: \"45d06a99-391d-46ef-8ae4-c6858cfa6511\") " pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-hostroot\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-sys-fs\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.929979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-run-ovn-kubernetes\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-cni-bin\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a995bb7d-7792-4c63-bbc1-ee1ec70f5986-agent-certs\") pod \"konnectivity-agent-jnj67\" (UID: \"a995bb7d-7792-4c63-bbc1-ee1ec70f5986\") " pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930091    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-var-lib-cni-multus\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-daemon-config\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-etc-selinux\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930215    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02b6ba59-b83b-4614-a80e-4802ed5d66d5-env-overrides\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-os-release\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.930985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-run-openvswitch\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.931915 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02b6ba59-b83b-4614-a80e-4802ed5d66d5-ovnkube-config\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:40.931915 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3429b6-f5f2-48ab-a368-28543e3860ad-tmp-dir\") pod \"node-resolver-ckcmv\" (UID: \"ed3429b6-f5f2-48ab-a368-28543e3860ad\") " pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:40.931915 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcb8000e-6bba-4dbc-a735-1022c0c72320-cni-binary-copy\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.931915 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-conf-dir\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:40.931915 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc3ae5c-574b-40bc-a2e5-24e47ffec047-host-slash\") pod \"iptables-alerter-f67ln\" (UID: \"8dc3ae5c-574b-40bc-a2e5-24e47ffec047\") " pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:40.931915 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a995bb7d-7792-4c63-bbc1-ee1ec70f5986-konnectivity-ca\") pod \"konnectivity-agent-jnj67\" (UID: \"a995bb7d-7792-4c63-bbc1-ee1ec70f5986\") " pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:02:40.931915 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.930419    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crpk4\" (UniqueName: \"kubernetes.io/projected/8dc3ae5c-574b-40bc-a2e5-24e47ffec047-kube-api-access-crpk4\") pod \"iptables-alerter-f67ln\" (UID: \"8dc3ae5c-574b-40bc-a2e5-24e47ffec047\") " pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:40.934199 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.934178    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:02:40.957004 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.956978    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bbbg8"
Jul 28 19:02:40.967513 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:40.966705    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bbbg8"
Jul 28 19:02:41.018254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.018171    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 19:02:41.031568 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-run-openvswitch\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.031735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02b6ba59-b83b-4614-a80e-4802ed5d66d5-ovnkube-config\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.031735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3429b6-f5f2-48ab-a368-28543e3860ad-tmp-dir\") pod \"node-resolver-ckcmv\" (UID: \"ed3429b6-f5f2-48ab-a368-28543e3860ad\") " pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:41.031735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcb8000e-6bba-4dbc-a735-1022c0c72320-cni-binary-copy\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.031735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-run-openvswitch\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.031735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-conf-dir\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.031735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc3ae5c-574b-40bc-a2e5-24e47ffec047-host-slash\") pod \"iptables-alerter-f67ln\" (UID: \"8dc3ae5c-574b-40bc-a2e5-24e47ffec047\") " pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:41.031735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a995bb7d-7792-4c63-bbc1-ee1ec70f5986-konnectivity-ca\") pod \"konnectivity-agent-jnj67\" (UID: \"a995bb7d-7792-4c63-bbc1-ee1ec70f5986\") " pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crpk4\" (UniqueName: \"kubernetes.io/projected/8dc3ae5c-574b-40bc-a2e5-24e47ffec047-kube-api-access-crpk4\") pod \"iptables-alerter-f67ln\" (UID: \"8dc3ae5c-574b-40bc-a2e5-24e47ffec047\") " pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-kubernetes\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-run\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcm9w\" (UniqueName: \"kubernetes.io/projected/7498e58d-3564-4246-ab55-92b6530ac4ae-kube-api-access-xcm9w\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-system-cni-dir\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7c6fda6c-c699-4ce6-a78a-209994e2db79-kubelet-config\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed3429b6-f5f2-48ab-a368-28543e3860ad-hosts-file\") pod \"node-resolver-ckcmv\" (UID: \"ed3429b6-f5f2-48ab-a368-28543e3860ad\") " pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.031978    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02b6ba59-b83b-4614-a80e-4802ed5d66d5-ovn-node-metrics-cert\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7c6fda6c-c699-4ce6-a78a-209994e2db79-dbus\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2w9f\" (UniqueName: \"kubernetes.io/projected/ed3429b6-f5f2-48ab-a368-28543e3860ad-kube-api-access-t2w9f\") pod \"node-resolver-ckcmv\" (UID: \"ed3429b6-f5f2-48ab-a368-28543e3860ad\") " pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-etc-kubernetes\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3429b6-f5f2-48ab-a368-28543e3860ad-tmp-dir\") pod \"node-resolver-ckcmv\" (UID: \"ed3429b6-f5f2-48ab-a368-28543e3860ad\") " pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-sysctl-conf\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032106    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4vxq\" (UniqueName: \"kubernetes.io/projected/b6e8d670-5c7a-4876-8249-85f13a045bab-kube-api-access-w4vxq\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.032127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-conf-dir\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-socket-dir-parent\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8dc3ae5c-574b-40bc-a2e5-24e47ffec047-iptables-alerter-script\") pod \"iptables-alerter-f67ln\" (UID: \"8dc3ae5c-574b-40bc-a2e5-24e47ffec047\") " pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032213    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-socket-dir-parent\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032224    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/645352cb-de5d-4b0b-88dc-670476f0deff-serviceca\") pod \"node-ca-9k5kh\" (UID: \"645352cb-de5d-4b0b-88dc-670476f0deff\") " pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-modprobe-d\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032260    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032278    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6e8d670-5c7a-4876-8249-85f13a045bab-cni-binary-copy\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02b6ba59-b83b-4614-a80e-4802ed5d66d5-ovnkube-config\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-registration-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032311    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7c6fda6c-c699-4ce6-a78a-209994e2db79-kubelet-config\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcb8000e-6bba-4dbc-a735-1022c0c72320-cni-binary-copy\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032370    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed3429b6-f5f2-48ab-a368-28543e3860ad-hosts-file\") pod \"node-resolver-ckcmv\" (UID: \"ed3429b6-f5f2-48ab-a368-28543e3860ad\") " pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032387    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dc3ae5c-574b-40bc-a2e5-24e47ffec047-host-slash\") pod \"iptables-alerter-f67ln\" (UID: \"8dc3ae5c-574b-40bc-a2e5-24e47ffec047\") " pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8dc3ae5c-574b-40bc-a2e5-24e47ffec047-iptables-alerter-script\") pod \"iptables-alerter-f67ln\" (UID: \"8dc3ae5c-574b-40bc-a2e5-24e47ffec047\") " pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-etc-kubernetes\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.032918 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032832    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032870    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-registration-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-kubelet\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-node-log\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7c6fda6c-c699-4ce6-a78a-209994e2db79-dbus\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032952    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-sysconfig\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.032994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-kubelet\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033027    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-node-log\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6e8d670-5c7a-4876-8249-85f13a045bab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/645352cb-de5d-4b0b-88dc-670476f0deff-host\") pod \"node-ca-9k5kh\" (UID: \"645352cb-de5d-4b0b-88dc-670476f0deff\") " pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-run-netns\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02b6ba59-b83b-4614-a80e-4802ed5d66d5-ovnkube-script-lib\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a995bb7d-7792-4c63-bbc1-ee1ec70f5986-konnectivity-ca\") pod \"konnectivity-agent-jnj67\" (UID: \"a995bb7d-7792-4c63-bbc1-ee1ec70f5986\") " pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033229    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhcjf\" (UniqueName: \"kubernetes.io/projected/645352cb-de5d-4b0b-88dc-670476f0deff-kube-api-access-dhcjf\") pod \"node-ca-9k5kh\" (UID: \"645352cb-de5d-4b0b-88dc-670476f0deff\") " pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-sysctl-d\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-run-netns\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.033776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033271    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-sys\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-var-lib-kubelet\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g256\" (UniqueName: \"kubernetes.io/projected/88ae7e56-364a-4f77-862a-2065daefbcfc-kube-api-access-5g256\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-cni-dir\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n2zk\" (UniqueName: \"kubernetes.io/projected/bcb8000e-6bba-4dbc-a735-1022c0c72320-kube-api-access-9n2zk\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-socket-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-systemd-units\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-cni-dir\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033482    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-log-socket\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-log-socket\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-cni-netd\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033559    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-cni-netd\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hp4z\" (UniqueName: \"kubernetes.io/projected/02b6ba59-b83b-4614-a80e-4802ed5d66d5-kube-api-access-4hp4z\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033574    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02b6ba59-b83b-4614-a80e-4802ed5d66d5-ovnkube-script-lib\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033608    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-systemd-units\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-var-lib-cni-bin\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-os-release\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.034574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-socket-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033689    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-slash\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-var-lib-cni-bin\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-run-k8s-cni-cncf-io\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-run-netns\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033794    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-slash\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-systemd\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-host\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033841    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-run-k8s-cni-cncf-io\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.033865    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033887    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-run-netns\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-system-cni-dir\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.033959    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret podName:7c6fda6c-c699-4ce6-a78a-209994e2db79 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:41.533916339 +0000 UTC m=+2.995524039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret") pod "global-pull-secret-syncer-wl5w9" (UID: "7c6fda6c-c699-4ce6-a78a-209994e2db79") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-system-cni-dir\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.033996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-run-multus-certs\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.034025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7498e58d-3564-4246-ab55-92b6530ac4ae-tmp\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.035241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.034071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6e8d670-5c7a-4876-8249-85f13a045bab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.036132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.034100    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-device-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.036132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.034101    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-run-multus-certs\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.036132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.034135    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8wrg\" (UniqueName: \"kubernetes.io/projected/0dff55ad-8180-488d-9996-3a9e46a31c44-kube-api-access-w8wrg\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.036132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.034521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-run-systemd\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.036544 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036522    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-device-dir\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.036594 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-etc-openvswitch\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.036636 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036618    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-cnibin\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.036673 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036652    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-var-lib-kubelet\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.036715 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-lib-modules\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.036750 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036716    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-var-lib-openvswitch\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.036788 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-run-ovn\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.036788 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwps\" (UniqueName: \"kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps\") pod \"network-check-target-29nx8\" (UID: \"45d06a99-391d-46ef-8ae4-c6858cfa6511\") " pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:41.036857 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-hostroot\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.036857 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-tuned\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.036974 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.036974 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-sys-fs\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.036974 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-run-ovn-kubernetes\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.036974 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.036970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-cni-bin\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.037192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a995bb7d-7792-4c63-bbc1-ee1ec70f5986-agent-certs\") pod \"konnectivity-agent-jnj67\" (UID: \"a995bb7d-7792-4c63-bbc1-ee1ec70f5986\") " pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:02:41.037192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:41.037192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037085    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-var-lib-cni-multus\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.037192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-daemon-config\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.037192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-cnibin\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.037192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-etc-selinux\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.037496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037207    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02b6ba59-b83b-4614-a80e-4802ed5d66d5-env-overrides\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.037496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-os-release\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.037496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-os-release\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.037496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-etc-openvswitch\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.037496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-cnibin\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.037750 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-var-lib-kubelet\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.037750 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-var-lib-openvswitch\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.037750 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.037642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-run-ovn\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.038754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.038168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-hostroot\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.038754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.038186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-cni-bin\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.038754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.038286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-etc-selinux\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.038754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.038345    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02b6ba59-b83b-4614-a80e-4802ed5d66d5-ovn-node-metrics-cert\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.038754 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.038474    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:41.038754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.038531    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02b6ba59-b83b-4614-a80e-4802ed5d66d5-env-overrides\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.038754 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.038541    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs podName:88ae7e56-364a-4f77-862a-2065daefbcfc nodeName:}" failed. No retries permitted until 2026-07-28 19:02:41.538513113 +0000 UTC m=+3.000120799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs") pod "network-metrics-daemon-9xt7v" (UID: "88ae7e56-364a-4f77-862a-2065daefbcfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:41.038754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.038624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0dff55ad-8180-488d-9996-3a9e46a31c44-sys-fs\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.038754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.038687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bcb8000e-6bba-4dbc-a735-1022c0c72320-multus-daemon-config\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.038754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.038688    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bcb8000e-6bba-4dbc-a735-1022c0c72320-host-var-lib-cni-multus\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.039277 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.038758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-host-run-ovn-kubernetes\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.039277 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.039015    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02b6ba59-b83b-4614-a80e-4802ed5d66d5-run-systemd\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.041472 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.041449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a995bb7d-7792-4c63-bbc1-ee1ec70f5986-agent-certs\") pod \"konnectivity-agent-jnj67\" (UID: \"a995bb7d-7792-4c63-bbc1-ee1ec70f5986\") " pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:02:41.044982 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.044958    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:41.044982 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.045002    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:41.045169 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.045017    2563 projected.go:196] Error preparing data for projected volume kube-api-access-xbwps for pod openshift-network-diagnostics/network-check-target-29nx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:41.045169 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.045096    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps podName:45d06a99-391d-46ef-8ae4-c6858cfa6511 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:41.545057086 +0000 UTC m=+3.006664772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xbwps" (UniqueName: "kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps") pod "network-check-target-29nx8" (UID: "45d06a99-391d-46ef-8ae4-c6858cfa6511") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:41.046553 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.046496    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2w9f\" (UniqueName: \"kubernetes.io/projected/ed3429b6-f5f2-48ab-a368-28543e3860ad-kube-api-access-t2w9f\") pod \"node-resolver-ckcmv\" (UID: \"ed3429b6-f5f2-48ab-a368-28543e3860ad\") " pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:41.047188 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.047169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g256\" (UniqueName: \"kubernetes.io/projected/88ae7e56-364a-4f77-862a-2065daefbcfc-kube-api-access-5g256\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:41.049080 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.049036    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crpk4\" (UniqueName: \"kubernetes.io/projected/8dc3ae5c-574b-40bc-a2e5-24e47ffec047-kube-api-access-crpk4\") pod \"iptables-alerter-f67ln\" (UID: \"8dc3ae5c-574b-40bc-a2e5-24e47ffec047\") " pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:41.050011 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.049975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n2zk\" (UniqueName: \"kubernetes.io/projected/bcb8000e-6bba-4dbc-a735-1022c0c72320-kube-api-access-9n2zk\") pod \"multus-lsrbm\" (UID: \"bcb8000e-6bba-4dbc-a735-1022c0c72320\") " pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.051600 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.051575    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8wrg\" (UniqueName: \"kubernetes.io/projected/0dff55ad-8180-488d-9996-3a9e46a31c44-kube-api-access-w8wrg\") pod \"aws-ebs-csi-driver-node-rc9dl\" (UID: \"0dff55ad-8180-488d-9996-3a9e46a31c44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.052673 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.052654    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hp4z\" (UniqueName: \"kubernetes.io/projected/02b6ba59-b83b-4614-a80e-4802ed5d66d5-kube-api-access-4hp4z\") pod \"ovnkube-node-62f22\" (UID: \"02b6ba59-b83b-4614-a80e-4802ed5d66d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.138007 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.137971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-sysconfig\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6e8d670-5c7a-4876-8249-85f13a045bab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.138192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138060    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/645352cb-de5d-4b0b-88dc-670476f0deff-host\") pod \"node-ca-9k5kh\" (UID: \"645352cb-de5d-4b0b-88dc-670476f0deff\") " pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:41.138192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138070    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-sysconfig\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhcjf\" (UniqueName: \"kubernetes.io/projected/645352cb-de5d-4b0b-88dc-670476f0deff-kube-api-access-dhcjf\") pod \"node-ca-9k5kh\" (UID: \"645352cb-de5d-4b0b-88dc-670476f0deff\") " pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:41.138192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-sysctl-d\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-sys\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-var-lib-kubelet\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138159    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/645352cb-de5d-4b0b-88dc-670476f0deff-host\") pod \"node-ca-9k5kh\" (UID: \"645352cb-de5d-4b0b-88dc-670476f0deff\") " pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:41.138192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-os-release\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138222    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-systemd\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-host\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-sys\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138305    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-sysctl-d\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-var-lib-kubelet\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-systemd\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-host\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-os-release\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7498e58d-3564-4246-ab55-92b6530ac4ae-tmp\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138527    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6e8d670-5c7a-4876-8249-85f13a045bab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-lib-modules\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-tuned\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.138676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138655    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-cnibin\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-kubernetes\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-run\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138827    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-lib-modules\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcm9w\" (UniqueName: \"kubernetes.io/projected/7498e58d-3564-4246-ab55-92b6530ac4ae-kube-api-access-xcm9w\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-system-cni-dir\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-sysctl-conf\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.138980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vxq\" (UniqueName: \"kubernetes.io/projected/b6e8d670-5c7a-4876-8249-85f13a045bab-kube-api-access-w4vxq\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.139044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/645352cb-de5d-4b0b-88dc-670476f0deff-serviceca\") pod \"node-ca-9k5kh\" (UID: \"645352cb-de5d-4b0b-88dc-670476f0deff\") " pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.139079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-modprobe-d\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.139106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6e8d670-5c7a-4876-8249-85f13a045bab-cni-binary-copy\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.139276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.139247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6e8d670-5c7a-4876-8249-85f13a045bab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.139678 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.139657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6e8d670-5c7a-4876-8249-85f13a045bab-cni-binary-copy\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.139750 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.139732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-system-cni-dir\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.139934 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.139889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-sysctl-conf\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.140161 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.140066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-modprobe-d\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.140161 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.140121    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-cnibin\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.140161 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.140175    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6e8d670-5c7a-4876-8249-85f13a045bab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.140434 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.140239    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-kubernetes\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.140434 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.140268    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6e8d670-5c7a-4876-8249-85f13a045bab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.140434 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.140292    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7498e58d-3564-4246-ab55-92b6530ac4ae-run\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.140434 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.140294    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/645352cb-de5d-4b0b-88dc-670476f0deff-serviceca\") pod \"node-ca-9k5kh\" (UID: \"645352cb-de5d-4b0b-88dc-670476f0deff\") " pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:41.142287 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.142192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7498e58d-3564-4246-ab55-92b6530ac4ae-etc-tuned\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.144595 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.144487    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7498e58d-3564-4246-ab55-92b6530ac4ae-tmp\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.144741 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.144583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal" event={"ID":"79fa5413215af614dae2a46132878cd3","Type":"ContainerStarted","Data":"32846caa5522f50cfe349020d6fefddc8e5c378daed88608846795fc30834970"}
Jul 28 19:02:41.146515 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.146486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal" event={"ID":"20fec549702e81f5927fb23efca53227","Type":"ContainerStarted","Data":"0736e637e2cc99bb467013b2be9e3e315cd04cc48a92d223ac815c2e118167bf"}
Jul 28 19:02:41.148458 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.148429    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhcjf\" (UniqueName: \"kubernetes.io/projected/645352cb-de5d-4b0b-88dc-670476f0deff-kube-api-access-dhcjf\") pod \"node-ca-9k5kh\" (UID: \"645352cb-de5d-4b0b-88dc-670476f0deff\") " pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:41.149714 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.149681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4vxq\" (UniqueName: \"kubernetes.io/projected/b6e8d670-5c7a-4876-8249-85f13a045bab-kube-api-access-w4vxq\") pod \"multus-additional-cni-plugins-8lhnl\" (UID: \"b6e8d670-5c7a-4876-8249-85f13a045bab\") " pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.153312 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.153291    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcm9w\" (UniqueName: \"kubernetes.io/projected/7498e58d-3564-4246-ab55-92b6530ac4ae-kube-api-access-xcm9w\") pod \"tuned-x7r6w\" (UID: \"7498e58d-3564-4246-ab55-92b6530ac4ae\") " pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.222049 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.222017    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl"
Jul 28 19:02:41.232381 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.232350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lsrbm"
Jul 28 19:02:41.240476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.240456    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:02:41.248470 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.248452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:02:41.257252 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.257232    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ckcmv"
Jul 28 19:02:41.265995 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.265969    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f67ln"
Jul 28 19:02:41.273648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.273600    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9k5kh"
Jul 28 19:02:41.280291 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.280261    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x7r6w"
Jul 28 19:02:41.288006 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.287987    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8lhnl"
Jul 28 19:02:41.542871 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.542785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:41.542871 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.542853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:41.543120 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.542990    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:41.543120 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.543057    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret podName:7c6fda6c-c699-4ce6-a78a-209994e2db79 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:42.54303554 +0000 UTC m=+4.004643227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret") pod "global-pull-secret-syncer-wl5w9" (UID: "7c6fda6c-c699-4ce6-a78a-209994e2db79") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:41.543396 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.542989    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:41.543492 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.543435    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs podName:88ae7e56-364a-4f77-862a-2065daefbcfc nodeName:}" failed. No retries permitted until 2026-07-28 19:02:42.543418725 +0000 UTC m=+4.005026409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs") pod "network-metrics-daemon-9xt7v" (UID: "88ae7e56-364a-4f77-862a-2065daefbcfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:41.644236 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.644202    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwps\" (UniqueName: \"kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps\") pod \"network-check-target-29nx8\" (UID: \"45d06a99-391d-46ef-8ae4-c6858cfa6511\") " pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:41.644416 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.644366    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:41.644416 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.644382    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:41.644416 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.644391    2563 projected.go:196] Error preparing data for projected volume kube-api-access-xbwps for pod openshift-network-diagnostics/network-check-target-29nx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:41.644549 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:41.644445    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps podName:45d06a99-391d-46ef-8ae4-c6858cfa6511 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:42.644426979 +0000 UTC m=+4.106034670 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xbwps" (UniqueName: "kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps") pod "network-check-target-29nx8" (UID: "45d06a99-391d-46ef-8ae4-c6858cfa6511") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:41.969339 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.969289    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 18:57:41 +0000 UTC" deadline="2028-02-14 20:33:45.629831666 +0000 UTC"
Jul 28 19:02:41.969339 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:41.969331    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13585h31m3.660505131s"
Jul 28 19:02:42.153689 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:42.153658    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7498e58d_3564_4246_ab55_92b6530ac4ae.slice/crio-5bbdd0cc2f8e2c6eb012c18b57d8a19da3d7fb98e42953ca906294e3ab550b92 WatchSource:0}: Error finding container 5bbdd0cc2f8e2c6eb012c18b57d8a19da3d7fb98e42953ca906294e3ab550b92: Status 404 returned error can't find the container with id 5bbdd0cc2f8e2c6eb012c18b57d8a19da3d7fb98e42953ca906294e3ab550b92
Jul 28 19:02:42.154400 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:42.154353    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod645352cb_de5d_4b0b_88dc_670476f0deff.slice/crio-acf3015a2f08f6015d82090dbabc569b81af0640b4995cfffec2ce8eb8a80126 WatchSource:0}: Error finding container acf3015a2f08f6015d82090dbabc569b81af0640b4995cfffec2ce8eb8a80126: Status 404 returned error can't find the container with id acf3015a2f08f6015d82090dbabc569b81af0640b4995cfffec2ce8eb8a80126
Jul 28 19:02:42.162111 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:42.162086    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dff55ad_8180_488d_9996_3a9e46a31c44.slice/crio-deaa3a8ed63d512bf4449815e019891b210558e2d1e2994cb9e7e29deadd9223 WatchSource:0}: Error finding container deaa3a8ed63d512bf4449815e019891b210558e2d1e2994cb9e7e29deadd9223: Status 404 returned error can't find the container with id deaa3a8ed63d512bf4449815e019891b210558e2d1e2994cb9e7e29deadd9223
Jul 28 19:02:42.162629 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:42.162602    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded3429b6_f5f2_48ab_a368_28543e3860ad.slice/crio-b2f9c43733bbcef14747c7c6d9d3ef58ce04061d0e889f9e910dbd78239b3baa WatchSource:0}: Error finding container b2f9c43733bbcef14747c7c6d9d3ef58ce04061d0e889f9e910dbd78239b3baa: Status 404 returned error can't find the container with id b2f9c43733bbcef14747c7c6d9d3ef58ce04061d0e889f9e910dbd78239b3baa
Jul 28 19:02:42.176301 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:02:42.175832    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e8d670_5c7a_4876_8249_85f13a045bab.slice/crio-c072e161e1e9c7f3b72699cf4afa34fd4dec819ae4befb8feba93dc076480065 WatchSource:0}: Error finding container c072e161e1e9c7f3b72699cf4afa34fd4dec819ae4befb8feba93dc076480065: Status 404 returned error can't find the container with id c072e161e1e9c7f3b72699cf4afa34fd4dec819ae4befb8feba93dc076480065
Jul 28 19:02:42.551216 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:42.551012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:42.551402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:42.551263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:42.551402 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:42.551190    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:42.551402 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:42.551354    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret podName:7c6fda6c-c699-4ce6-a78a-209994e2db79 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:44.551337014 +0000 UTC m=+6.012944717 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret") pod "global-pull-secret-syncer-wl5w9" (UID: "7c6fda6c-c699-4ce6-a78a-209994e2db79") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:42.551402 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:42.551397    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:42.551582 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:42.551459    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs podName:88ae7e56-364a-4f77-862a-2065daefbcfc nodeName:}" failed. No retries permitted until 2026-07-28 19:02:44.551441641 +0000 UTC m=+6.013049325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs") pod "network-metrics-daemon-9xt7v" (UID: "88ae7e56-364a-4f77-862a-2065daefbcfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:42.651992 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:42.651901    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwps\" (UniqueName: \"kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps\") pod \"network-check-target-29nx8\" (UID: \"45d06a99-391d-46ef-8ae4-c6858cfa6511\") " pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:42.652156 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:42.652138    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:42.652218 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:42.652163    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:42.652218 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:42.652176    2563 projected.go:196] Error preparing data for projected volume kube-api-access-xbwps for pod openshift-network-diagnostics/network-check-target-29nx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:42.652317 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:42.652237    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps podName:45d06a99-391d-46ef-8ae4-c6858cfa6511 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:44.652217567 +0000 UTC m=+6.113825250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xbwps" (UniqueName: "kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps") pod "network-check-target-29nx8" (UID: "45d06a99-391d-46ef-8ae4-c6858cfa6511") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:42.970078 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:42.969964    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 18:57:41 +0000 UTC" deadline="2028-03-04 14:47:22.735012125 +0000 UTC"
Jul 28 19:02:42.970078 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:42.970010    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14035h44m39.765005844s"
Jul 28 19:02:43.139909 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.139122    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:43.139909 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:43.139273    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:02:43.139909 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.139698    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:43.139909 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:43.139785    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:02:43.140473 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.140329    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:43.140473 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:43.140427    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:02:43.163266 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.163205    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" event={"ID":"02b6ba59-b83b-4614-a80e-4802ed5d66d5","Type":"ContainerStarted","Data":"5d14c48f12787e08ba43a47b09f4430cb31d4bb42fbef38b4eb3f62813cc8b49"}
Jul 28 19:02:43.183255 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.183080    2563 generic.go:356] "Generic (PLEG): container finished" podID="79fa5413215af614dae2a46132878cd3" containerID="449310dd2a2a22cec494c15b34a1ceb8f84d9275b90227ead27edc6fcda41abc" exitCode=0
Jul 28 19:02:43.183255 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.183182    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal" event={"ID":"79fa5413215af614dae2a46132878cd3","Type":"ContainerDied","Data":"449310dd2a2a22cec494c15b34a1ceb8f84d9275b90227ead27edc6fcda41abc"}
Jul 28 19:02:43.196945 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.196901    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal" event={"ID":"20fec549702e81f5927fb23efca53227","Type":"ContainerStarted","Data":"bac8e67de604b8c04a36f452950360dd98cdbe6ac25da5dd137496e54d92e5b5"}
Jul 28 19:02:43.206168 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.206078    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8lhnl" event={"ID":"b6e8d670-5c7a-4876-8249-85f13a045bab","Type":"ContainerStarted","Data":"c072e161e1e9c7f3b72699cf4afa34fd4dec819ae4befb8feba93dc076480065"}
Jul 28 19:02:43.214563 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.214507    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal" podStartSLOduration=3.214485836 podStartE2EDuration="3.214485836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:43.214335729 +0000 UTC m=+4.675943437" watchObservedRunningTime="2026-07-28 19:02:43.214485836 +0000 UTC m=+4.676093546"
Jul 28 19:02:43.217211 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.217106    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl" event={"ID":"0dff55ad-8180-488d-9996-3a9e46a31c44","Type":"ContainerStarted","Data":"deaa3a8ed63d512bf4449815e019891b210558e2d1e2994cb9e7e29deadd9223"}
Jul 28 19:02:43.221273 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.221154    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9k5kh" event={"ID":"645352cb-de5d-4b0b-88dc-670476f0deff","Type":"ContainerStarted","Data":"acf3015a2f08f6015d82090dbabc569b81af0640b4995cfffec2ce8eb8a80126"}
Jul 28 19:02:43.227581 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.227522    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x7r6w" event={"ID":"7498e58d-3564-4246-ab55-92b6530ac4ae","Type":"ContainerStarted","Data":"5bbdd0cc2f8e2c6eb012c18b57d8a19da3d7fb98e42953ca906294e3ab550b92"}
Jul 28 19:02:43.232140 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.232075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jnj67" event={"ID":"a995bb7d-7792-4c63-bbc1-ee1ec70f5986","Type":"ContainerStarted","Data":"0513938ff7038de5817d9aea1082a9b465bf5a3842056612229ef0b1ffaf1c9a"}
Jul 28 19:02:43.240111 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.240083    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f67ln" event={"ID":"8dc3ae5c-574b-40bc-a2e5-24e47ffec047","Type":"ContainerStarted","Data":"798b1860077e1f6d96f323985ca1948506ae19c2864acd551d530d2127a4f34c"}
Jul 28 19:02:43.248869 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.248802    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lsrbm" event={"ID":"bcb8000e-6bba-4dbc-a735-1022c0c72320","Type":"ContainerStarted","Data":"a6a7d4c2cb5ba753d6159cb4588b76e78696e6cbe555b660434efc0dbbb845a2"}
Jul 28 19:02:43.253803 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:43.253706    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ckcmv" event={"ID":"ed3429b6-f5f2-48ab-a368-28543e3860ad","Type":"ContainerStarted","Data":"b2f9c43733bbcef14747c7c6d9d3ef58ce04061d0e889f9e910dbd78239b3baa"}
Jul 28 19:02:44.318110 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:44.318068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal" event={"ID":"79fa5413215af614dae2a46132878cd3","Type":"ContainerStarted","Data":"d936490ce090f133f3d8466d0a896262402530e39cf4a8faf1422a857ba7e64c"}
Jul 28 19:02:44.571635 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:44.571544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:44.571635 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:44.571614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:44.571859 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:44.571734    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:44.571859 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:44.571787    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs podName:88ae7e56-364a-4f77-862a-2065daefbcfc nodeName:}" failed. No retries permitted until 2026-07-28 19:02:48.571773011 +0000 UTC m=+10.033380694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs") pod "network-metrics-daemon-9xt7v" (UID: "88ae7e56-364a-4f77-862a-2065daefbcfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:44.572285 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:44.572173    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:44.572285 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:44.572235    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret podName:7c6fda6c-c699-4ce6-a78a-209994e2db79 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:48.572217135 +0000 UTC m=+10.033824821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret") pod "global-pull-secret-syncer-wl5w9" (UID: "7c6fda6c-c699-4ce6-a78a-209994e2db79") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:44.672616 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:44.672562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwps\" (UniqueName: \"kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps\") pod \"network-check-target-29nx8\" (UID: \"45d06a99-391d-46ef-8ae4-c6858cfa6511\") " pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:44.672787 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:44.672773    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:44.672854 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:44.672797    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:44.672854 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:44.672811    2563 projected.go:196] Error preparing data for projected volume kube-api-access-xbwps for pod openshift-network-diagnostics/network-check-target-29nx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:44.672977 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:44.672877    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps podName:45d06a99-391d-46ef-8ae4-c6858cfa6511 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:48.672856334 +0000 UTC m=+10.134464041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xbwps" (UniqueName: "kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps") pod "network-check-target-29nx8" (UID: "45d06a99-391d-46ef-8ae4-c6858cfa6511") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:45.139097 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:45.139061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:45.139293 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:45.139213    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:02:45.139656 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:45.139067    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:45.139861 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:45.139802    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:02:45.139935 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:45.139893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:45.140030 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:45.139983    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:02:47.138558 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:47.138040    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:47.138558 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:47.138052    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:47.138558 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:47.138165    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:02:47.138558 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:47.138246    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:02:47.138558 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:47.138052    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:47.138558 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:47.138334    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:02:48.606837 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:48.606801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:48.607331 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:48.606860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:48.607331 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:48.607014    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:48.607331 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:48.607066    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs podName:88ae7e56-364a-4f77-862a-2065daefbcfc nodeName:}" failed. No retries permitted until 2026-07-28 19:02:56.607051661 +0000 UTC m=+18.068659345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs") pod "network-metrics-daemon-9xt7v" (UID: "88ae7e56-364a-4f77-862a-2065daefbcfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:48.607494 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:48.607468    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:48.607551 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:48.607523    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret podName:7c6fda6c-c699-4ce6-a78a-209994e2db79 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:56.607506021 +0000 UTC m=+18.069113709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret") pod "global-pull-secret-syncer-wl5w9" (UID: "7c6fda6c-c699-4ce6-a78a-209994e2db79") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:48.707714 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:48.707676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwps\" (UniqueName: \"kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps\") pod \"network-check-target-29nx8\" (UID: \"45d06a99-391d-46ef-8ae4-c6858cfa6511\") " pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:48.707892 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:48.707830    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:48.707892 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:48.707856    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:48.707892 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:48.707868    2563 projected.go:196] Error preparing data for projected volume kube-api-access-xbwps for pod openshift-network-diagnostics/network-check-target-29nx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:48.708083 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:48.707933    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps podName:45d06a99-391d-46ef-8ae4-c6858cfa6511 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:56.707906819 +0000 UTC m=+18.169514500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xbwps" (UniqueName: "kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps") pod "network-check-target-29nx8" (UID: "45d06a99-391d-46ef-8ae4-c6858cfa6511") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:49.138752 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:49.138718    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:49.138932 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:49.138863    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:02:49.138932 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:49.138886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:49.139057 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:49.138886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:49.139057 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:49.138996    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:02:49.139154 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:49.139094    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:02:51.139226 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:51.138516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:51.139226 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:51.138540    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:51.139226 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:51.138634    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:02:51.139226 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:51.138988    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:51.139226 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:51.139076    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:02:51.139226 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:51.139155    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:02:53.138593 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:53.138513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:53.139064 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:53.138511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:53.139064 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:53.138664    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:02:53.139064 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:53.138513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:53.139064 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:53.138766    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:02:53.139064 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:53.138832    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:02:55.138364 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:55.138310    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:55.138364 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:55.138329    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:55.138817 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:55.138441    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:02:55.138817 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:55.138480    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:55.138817 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:55.138551    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:02:55.138817 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:55.138591    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:02:56.670402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:56.670364    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:56.670851 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:56.670446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:56.670851 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:56.670530    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:56.670851 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:56.670544    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:56.670851 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:56.670604    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret podName:7c6fda6c-c699-4ce6-a78a-209994e2db79 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:12.670583994 +0000 UTC m=+34.132191680 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret") pod "global-pull-secret-syncer-wl5w9" (UID: "7c6fda6c-c699-4ce6-a78a-209994e2db79") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:56.670851 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:56.670622    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs podName:88ae7e56-364a-4f77-862a-2065daefbcfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:12.67061504 +0000 UTC m=+34.132222721 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs") pod "network-metrics-daemon-9xt7v" (UID: "88ae7e56-364a-4f77-862a-2065daefbcfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:56.771102 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:56.771050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwps\" (UniqueName: \"kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps\") pod \"network-check-target-29nx8\" (UID: \"45d06a99-391d-46ef-8ae4-c6858cfa6511\") " pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:56.771269 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:56.771211    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:56.771269 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:56.771230    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:56.771269 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:56.771240    2563 projected.go:196] Error preparing data for projected volume kube-api-access-xbwps for pod openshift-network-diagnostics/network-check-target-29nx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:56.771384 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:56.771303    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps podName:45d06a99-391d-46ef-8ae4-c6858cfa6511 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:12.771284533 +0000 UTC m=+34.232892221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xbwps" (UniqueName: "kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps") pod "network-check-target-29nx8" (UID: "45d06a99-391d-46ef-8ae4-c6858cfa6511") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:57.138330 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:57.138291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:57.138330 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:57.138317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:57.138545 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:57.138367    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:57.138545 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:57.138453    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:02:57.138545 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:57.138506    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:02:57.138642 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:57.138579    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:02:59.139532 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:59.139088    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:02:59.139532 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:59.139181    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:02:59.139532 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:59.139196    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:02:59.139532 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:02:59.139233    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:02:59.139532 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:59.139340    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:02:59.139532 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:02:59.139397    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:03:01.137973 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.137905    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:01.138626 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.137914    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:01.138626 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:01.138057    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:03:01.138626 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.138073    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:01.138626 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:01.138185    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:03:01.138626 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:01.138280    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:03:01.351188 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.351047    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6e8d670-5c7a-4876-8249-85f13a045bab" containerID="3c79c02b82d1e01f171ef76f803614afacbcd37f6a8bf08f26a1d4a86d05927c" exitCode=0
Jul 28 19:03:01.351341 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.351112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8lhnl" event={"ID":"b6e8d670-5c7a-4876-8249-85f13a045bab","Type":"ContainerDied","Data":"3c79c02b82d1e01f171ef76f803614afacbcd37f6a8bf08f26a1d4a86d05927c"}
Jul 28 19:03:01.352779 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.352757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl" event={"ID":"0dff55ad-8180-488d-9996-3a9e46a31c44","Type":"ContainerStarted","Data":"5c0ccb1b0845248e4649753d24a1cf2b04e0d3286b19411d12c18efcccbf0e60"}
Jul 28 19:03:01.354061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.354036    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9k5kh" event={"ID":"645352cb-de5d-4b0b-88dc-670476f0deff","Type":"ContainerStarted","Data":"24e825e265254743689b72b5963ee7b7b796ba294c3acf2fe44e6a8b716cd2be"}
Jul 28 19:03:01.355480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.355462    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x7r6w" event={"ID":"7498e58d-3564-4246-ab55-92b6530ac4ae","Type":"ContainerStarted","Data":"f46f36b8c5c42a141b27737a16078d32370bd30f3eb72d7832058a0d62f89ec9"}
Jul 28 19:03:01.356754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.356729    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jnj67" event={"ID":"a995bb7d-7792-4c63-bbc1-ee1ec70f5986","Type":"ContainerStarted","Data":"b55d8fea754e97afa9c728f5659a33e3eadac5295d91017eacecb08b6ecb0e3f"}
Jul 28 19:03:01.358142 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.358114    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lsrbm" event={"ID":"bcb8000e-6bba-4dbc-a735-1022c0c72320","Type":"ContainerStarted","Data":"5be39fc9620eee03d4aebafc79d36e54e1f1ab564acff23e903febdc2c9fcd6b"}
Jul 28 19:03:01.359502 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.359485    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ckcmv" event={"ID":"ed3429b6-f5f2-48ab-a368-28543e3860ad","Type":"ContainerStarted","Data":"354afde27833218e8a5a996cb8549d6dde55967b06fa462abd1d3ec0237cd6c0"}
Jul 28 19:03:01.361166 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.361151    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:03:01.361437 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.361422    2563 generic.go:356] "Generic (PLEG): container finished" podID="02b6ba59-b83b-4614-a80e-4802ed5d66d5" containerID="574b9064770f117cc9797ebaff62fb38ad59a2e2251106d81e203663ef0b60f9" exitCode=1
Jul 28 19:03:01.361483 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.361464    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" event={"ID":"02b6ba59-b83b-4614-a80e-4802ed5d66d5","Type":"ContainerStarted","Data":"be4d0b79395e0b2aff625913900c72988b6ab652e78bb822035262f66bbe4ccc"}
Jul 28 19:03:01.361520 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.361507    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" event={"ID":"02b6ba59-b83b-4614-a80e-4802ed5d66d5","Type":"ContainerDied","Data":"574b9064770f117cc9797ebaff62fb38ad59a2e2251106d81e203663ef0b60f9"}
Jul 28 19:03:01.361549 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.361537    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" event={"ID":"02b6ba59-b83b-4614-a80e-4802ed5d66d5","Type":"ContainerStarted","Data":"1587757a78c9d590b4d1d23ce7de6fc01eba2848324224dbbf33672ea8f09570"}
Jul 28 19:03:01.376914 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.376880    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-64.ec2.internal" podStartSLOduration=21.376868046 podStartE2EDuration="21.376868046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:44.338192733 +0000 UTC m=+5.799800440" watchObservedRunningTime="2026-07-28 19:03:01.376868046 +0000 UTC m=+22.838475748"
Jul 28 19:03:01.390673 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.390636    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9k5kh" podStartSLOduration=3.696567432 podStartE2EDuration="22.39062271s" totalImagesPullingTime="18.694055278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:01.390374767 +0000 UTC m=+22.851982471" watchObservedRunningTime="2026-07-28 19:03:01.39062271 +0000 UTC m=+22.852230412"
Jul 28 19:03:01.406284 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.406235    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x7r6w" podStartSLOduration=3.712301452 podStartE2EDuration="22.406215996s" totalImagesPullingTime="18.693914544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:01.405860051 +0000 UTC m=+22.867467754" watchObservedRunningTime="2026-07-28 19:03:01.406215996 +0000 UTC m=+22.867823700"
Jul 28 19:03:01.448797 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.448705    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lsrbm" podStartSLOduration=3.733794466 podStartE2EDuration="22.448688344s" totalImagesPullingTime="18.714893878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:01.43181018 +0000 UTC m=+22.893417882" watchObservedRunningTime="2026-07-28 19:03:01.448688344 +0000 UTC m=+22.910296048"
Jul 28 19:03:01.449231 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.449206    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ckcmv" podStartSLOduration=3.788854743 podStartE2EDuration="22.449199437s" totalImagesPullingTime="18.660344694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:01.448818981 +0000 UTC m=+22.910426666" watchObservedRunningTime="2026-07-28 19:03:01.449199437 +0000 UTC m=+22.910807140"
Jul 28 19:03:01.463238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.463208    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jnj67" podStartSLOduration=8.058131105 podStartE2EDuration="22.463195472s" totalImagesPullingTime="14.405064367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:01.463123949 +0000 UTC m=+22.924731651" watchObservedRunningTime="2026-07-28 19:03:01.463195472 +0000 UTC m=+22.924803175"
Jul 28 19:03:01.673457 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.673424    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:01.735855 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.735827    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:01.977198 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:01.977122    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:02.044034 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.044005    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:02.366121 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.366082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f67ln" event={"ID":"8dc3ae5c-574b-40bc-a2e5-24e47ffec047","Type":"ContainerStarted","Data":"5d1c3e3d2b061b00b7cbdae3350cad7f6a69c2063a6b106fd7233ce3ac48d99e"}
Jul 28 19:03:02.369898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.369878    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:03:02.370369 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.370343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" event={"ID":"02b6ba59-b83b-4614-a80e-4802ed5d66d5","Type":"ContainerStarted","Data":"e8fd0332919e500bbf87304e132269a4c709a924d78a9b74eda4f0f04cacd2e8"}
Jul 28 19:03:02.370480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.370421    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" event={"ID":"02b6ba59-b83b-4614-a80e-4802ed5d66d5","Type":"ContainerStarted","Data":"85cd0c23d3d4a5b293a2300959e844e16393f6cab5937580c44af95569e6ab8a"}
Jul 28 19:03:02.370480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.370463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" event={"ID":"02b6ba59-b83b-4614-a80e-4802ed5d66d5","Type":"ContainerStarted","Data":"7f8b62388582baed79d7c664057ed0d035f0bc482e79de772d111f1a0eeb76ef"}
Jul 28 19:03:02.382809 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.382769    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f67ln" podStartSLOduration=4.705314082 podStartE2EDuration="23.382750443s" totalImagesPullingTime="18.677436361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:02.382490394 +0000 UTC m=+23.844098098" watchObservedRunningTime="2026-07-28 19:03:02.382750443 +0000 UTC m=+23.844358151"
Jul 28 19:03:02.425002 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.424971    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 19:03:02.450507 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.450481    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:03:02.451024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.451004    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:03:02.492559 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.492526    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:02.554213 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.554136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:02.995192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.995072    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T19:03:02.424996667Z","UUID":"b7dce90d-40b0-4fac-9b1f-c9d4cab710b4","Handler":null,"Name":"","Endpoint":""}
Jul 28 19:03:02.997146 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.997121    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 19:03:02.997276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:02.997168    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 19:03:03.139066 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:03.139035    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:03.139236 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:03.139162    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:03:03.139616 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:03.139595    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:03.139729 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:03.139682    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:03:03.141020 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:03.141002    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:03.141111 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:03.141093    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:03:03.377734 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:03.377698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl" event={"ID":"0dff55ad-8180-488d-9996-3a9e46a31c44","Type":"ContainerStarted","Data":"dc9bf2917b7d88d86069fc6e1cf67fcd24e31db30f0a8e057698c74414997faa"}
Jul 28 19:03:04.382301 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:04.382265    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl" event={"ID":"0dff55ad-8180-488d-9996-3a9e46a31c44","Type":"ContainerStarted","Data":"5dcbd088b27d80c994090288689017467208e7ab3a38479fae09d6dd9fee2259"}
Jul 28 19:03:04.385903 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:04.385881    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:03:04.386265 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:04.386232    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" event={"ID":"02b6ba59-b83b-4614-a80e-4802ed5d66d5","Type":"ContainerStarted","Data":"bf454f78c93fb190a5f28eb35d9d7b4b354d016f87a848e7b70ebf6b900b27de"}
Jul 28 19:03:04.386265 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:04.386257    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 19:03:04.400689 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:04.400638    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rc9dl" podStartSLOduration=4.552916182 podStartE2EDuration="25.400618925s" totalImagesPullingTime="20.847702743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:04.399784511 +0000 UTC m=+25.861392216" watchObservedRunningTime="2026-07-28 19:03:04.400618925 +0000 UTC m=+25.862226629"
Jul 28 19:03:05.138442 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:05.138209    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:05.138618 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:05.138267    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:05.138618 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:05.138536    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:03:05.138746 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:05.138633    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:03:05.138746 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:05.138275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:05.138831 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:05.138746    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:03:06.393100 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:06.393075    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:03:06.393865 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:06.393472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" event={"ID":"02b6ba59-b83b-4614-a80e-4802ed5d66d5","Type":"ContainerStarted","Data":"3b3bd576b6f688614d7d1e61528775f0b7270e2fb5de08ea11de668f31f9e52f"}
Jul 28 19:03:06.393865 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:06.393710    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:03:06.394040 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:06.393967    2563 scope.go:117] "RemoveContainer" containerID="574b9064770f117cc9797ebaff62fb38ad59a2e2251106d81e203663ef0b60f9"
Jul 28 19:03:06.395671 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:06.395631    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6e8d670-5c7a-4876-8249-85f13a045bab" containerID="d8e958cc43ca625985bfbc10b229fcbb28d26348ab5d1d97b486480ab46f2542" exitCode=0
Jul 28 19:03:06.395760 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:06.395690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8lhnl" event={"ID":"b6e8d670-5c7a-4876-8249-85f13a045bab","Type":"ContainerDied","Data":"d8e958cc43ca625985bfbc10b229fcbb28d26348ab5d1d97b486480ab46f2542"}
Jul 28 19:03:06.412706 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:06.412683    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:03:07.137809 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.137644    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:07.137931 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:07.137894    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:03:07.137931 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.137707    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:07.138051 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:07.137976    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:03:07.138051 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.137686    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:07.138149 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:07.138109    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:03:07.400167 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.400139    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6e8d670-5c7a-4876-8249-85f13a045bab" containerID="79d92008a28e712255cb9347035dff1e6bb781f590eebaf2e0599a3a8b82e5fe" exitCode=0
Jul 28 19:03:07.400565 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.400232    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8lhnl" event={"ID":"b6e8d670-5c7a-4876-8249-85f13a045bab","Type":"ContainerDied","Data":"79d92008a28e712255cb9347035dff1e6bb781f590eebaf2e0599a3a8b82e5fe"}
Jul 28 19:03:07.404531 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.404515    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:03:07.404873 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.404854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" event={"ID":"02b6ba59-b83b-4614-a80e-4802ed5d66d5","Type":"ContainerStarted","Data":"1a6e12ca938820925c4bf12813687be76d2297c292a5796b6bdd97ab4db77ad7"}
Jul 28 19:03:07.405248 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.405233    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:03:07.405373 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.405294    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:03:07.421353 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.421330    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:03:07.458019 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:07.457976    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-62f22" podStartSLOduration=9.699286564 podStartE2EDuration="28.457961716s" totalImagesPullingTime="18.758675152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:07.456193507 +0000 UTC m=+28.917801209" watchObservedRunningTime="2026-07-28 19:03:07.457961716 +0000 UTC m=+28.919569419"
Jul 28 19:03:08.000448 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:08.000349    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-29nx8"]
Jul 28 19:03:08.000539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:08.000482    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:08.000613 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:08.000585    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:03:08.002789 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:08.002769    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9xt7v"]
Jul 28 19:03:08.002911 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:08.002874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:08.002911 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:08.002904    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wl5w9"]
Jul 28 19:03:08.003065 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:08.003001    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:03:08.003065 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:08.003039    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:08.003185 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:08.003099    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:03:08.409777 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:08.409750    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6e8d670-5c7a-4876-8249-85f13a045bab" containerID="b9a9db9175f937cae2c31ac7613ae7b7efc8c483b50048a1bd5709f8e85ced0a" exitCode=0
Jul 28 19:03:08.410204 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:08.409832    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8lhnl" event={"ID":"b6e8d670-5c7a-4876-8249-85f13a045bab","Type":"ContainerDied","Data":"b9a9db9175f937cae2c31ac7613ae7b7efc8c483b50048a1bd5709f8e85ced0a"}
Jul 28 19:03:08.629691 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:08.629662    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:09.997786 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:09.997753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:10.138036 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:10.137959    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:10.138193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:10.137962    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:10.138193 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:10.138104    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:03:10.138316 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:10.138190    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:03:10.138316 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:10.137960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:10.138316 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:10.138286    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:03:10.172784 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:10.172746    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:03:10.173264 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:10.172886    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 19:03:10.173364 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:10.173345    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jnj67"
Jul 28 19:03:10.242540 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:10.242494    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:12.138745 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:12.138356    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:12.138745 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:12.138373    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:12.138745 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:12.138468    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:12.138745 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.138486    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29nx8" podUID="45d06a99-391d-46ef-8ae4-c6858cfa6511"
Jul 28 19:03:12.138745 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.138588    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl5w9" podUID="7c6fda6c-c699-4ce6-a78a-209994e2db79"
Jul 28 19:03:12.138745 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.138686    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xt7v" podUID="88ae7e56-364a-4f77-862a-2065daefbcfc"
Jul 28 19:03:12.427943 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:12.427691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:12.490165 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:12.490133    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:12.690483 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:12.690401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:12.690483 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:12.690460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:12.690675 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.690551    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:03:12.690675 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.690557    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:03:12.690675 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.690612    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs podName:88ae7e56-364a-4f77-862a-2065daefbcfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:44.690597614 +0000 UTC m=+66.152205295 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs") pod "network-metrics-daemon-9xt7v" (UID: "88ae7e56-364a-4f77-862a-2065daefbcfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:03:12.690675 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.690626    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret podName:7c6fda6c-c699-4ce6-a78a-209994e2db79 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:44.69062006 +0000 UTC m=+66.152227742 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret") pod "global-pull-secret-syncer-wl5w9" (UID: "7c6fda6c-c699-4ce6-a78a-209994e2db79") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:03:12.791183 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:12.791152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwps\" (UniqueName: \"kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps\") pod \"network-check-target-29nx8\" (UID: \"45d06a99-391d-46ef-8ae4-c6858cfa6511\") " pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:12.791343 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.791291    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:03:12.791343 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.791306    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:03:12.791343 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.791315    2563 projected.go:196] Error preparing data for projected volume kube-api-access-xbwps for pod openshift-network-diagnostics/network-check-target-29nx8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:03:12.791457 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:12.791363    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps podName:45d06a99-391d-46ef-8ae4-c6858cfa6511 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:44.791351464 +0000 UTC m=+66.252959149 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xbwps" (UniqueName: "kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps") pod "network-check-target-29nx8" (UID: "45d06a99-391d-46ef-8ae4-c6858cfa6511") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:03:13.048220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.048185    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:13.135003 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.134980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:13.425039 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.425007    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6e8d670-5c7a-4876-8249-85f13a045bab" containerID="cb056d8b038c1d9477eca8703942de2b3ec67c5f2a56f21bc2a4189a2c3f5081" exitCode=0
Jul 28 19:03:13.425404 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.425086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8lhnl" event={"ID":"b6e8d670-5c7a-4876-8249-85f13a045bab","Type":"ContainerDied","Data":"cb056d8b038c1d9477eca8703942de2b3ec67c5f2a56f21bc2a4189a2c3f5081"}
Jul 28 19:03:13.822643 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.822615    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-64.ec2.internal" event="NodeReady"
Jul 28 19:03:13.822905 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.822738    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 19:03:13.866146 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.866118    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-smkjm"]
Jul 28 19:03:13.882943 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.882897    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mj6fw"]
Jul 28 19:03:13.883098 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.883019    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:13.885690 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.885548    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 19:03:13.885870 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.885852    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d4lpd\""
Jul 28 19:03:13.886192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.886172    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 19:03:13.901379 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.901359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-smkjm"]
Jul 28 19:03:13.901501 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.901392    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mj6fw"]
Jul 28 19:03:13.901501 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.901486    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mj6fw"
Jul 28 19:03:13.903946 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.903910    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 19:03:13.904100 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.903953    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nww2p\""
Jul 28 19:03:13.904100 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.904036    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 19:03:13.904213 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.904197    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 19:03:13.998530 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.998502    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4wct\" (UniqueName: \"kubernetes.io/projected/1cf35915-6763-44f1-8d21-5ef270aad4f8-kube-api-access-b4wct\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:13.998701 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.998552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cf35915-6763-44f1-8d21-5ef270aad4f8-tmp-dir\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:13.998701 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.998580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/daa9a9e1-99e7-4bd1-9745-9231f6f83ba5-cert\") pod \"ingress-canary-mj6fw\" (UID: \"daa9a9e1-99e7-4bd1-9745-9231f6f83ba5\") " pod="openshift-ingress-canary/ingress-canary-mj6fw"
Jul 28 19:03:13.998701 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.998596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1cf35915-6763-44f1-8d21-5ef270aad4f8-config-volume\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:13.998701 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.998635    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ksjf\" (UniqueName: \"kubernetes.io/projected/daa9a9e1-99e7-4bd1-9745-9231f6f83ba5-kube-api-access-8ksjf\") pod \"ingress-canary-mj6fw\" (UID: \"daa9a9e1-99e7-4bd1-9745-9231f6f83ba5\") " pod="openshift-ingress-canary/ingress-canary-mj6fw"
Jul 28 19:03:13.998701 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.998679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:13.999958 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:13.999908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:14.099177 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.099098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ksjf\" (UniqueName: \"kubernetes.io/projected/daa9a9e1-99e7-4bd1-9745-9231f6f83ba5-kube-api-access-8ksjf\") pod \"ingress-canary-mj6fw\" (UID: \"daa9a9e1-99e7-4bd1-9745-9231f6f83ba5\") " pod="openshift-ingress-canary/ingress-canary-mj6fw"
Jul 28 19:03:14.099177 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.099132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:14.099177 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.099152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wct\" (UniqueName: \"kubernetes.io/projected/1cf35915-6763-44f1-8d21-5ef270aad4f8-kube-api-access-b4wct\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:14.099518 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.099188    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cf35915-6763-44f1-8d21-5ef270aad4f8-tmp-dir\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:14.099518 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.099214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/daa9a9e1-99e7-4bd1-9745-9231f6f83ba5-cert\") pod \"ingress-canary-mj6fw\" (UID: \"daa9a9e1-99e7-4bd1-9745-9231f6f83ba5\") " pod="openshift-ingress-canary/ingress-canary-mj6fw"
Jul 28 19:03:14.099518 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.099236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1cf35915-6763-44f1-8d21-5ef270aad4f8-config-volume\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:14.099518 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:14.099269    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:14.099518 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:14.099337    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls podName:1cf35915-6763-44f1-8d21-5ef270aad4f8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:14.599318058 +0000 UTC m=+36.060925740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls") pod "dns-default-smkjm" (UID: "1cf35915-6763-44f1-8d21-5ef270aad4f8") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:14.099789 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.099634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cf35915-6763-44f1-8d21-5ef270aad4f8-tmp-dir\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:14.099840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.099810    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1cf35915-6763-44f1-8d21-5ef270aad4f8-config-volume\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:14.100112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.100092    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:14.104115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.104096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/daa9a9e1-99e7-4bd1-9745-9231f6f83ba5-cert\") pod \"ingress-canary-mj6fw\" (UID: \"daa9a9e1-99e7-4bd1-9745-9231f6f83ba5\") " pod="openshift-ingress-canary/ingress-canary-mj6fw"
Jul 28 19:03:14.109486 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.109468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4wct\" (UniqueName: \"kubernetes.io/projected/1cf35915-6763-44f1-8d21-5ef270aad4f8-kube-api-access-b4wct\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:14.109738 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.109722    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ksjf\" (UniqueName: \"kubernetes.io/projected/daa9a9e1-99e7-4bd1-9745-9231f6f83ba5-kube-api-access-8ksjf\") pod \"ingress-canary-mj6fw\" (UID: \"daa9a9e1-99e7-4bd1-9745-9231f6f83ba5\") " pod="openshift-ingress-canary/ingress-canary-mj6fw"
Jul 28 19:03:14.138181 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.138164    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:14.138181 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.138176    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:14.138324 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.138271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:14.141005 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.140988    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:03:14.141108 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.141052    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:03:14.141108 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.141071    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:03:14.141222 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.141172    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-snpbv\""
Jul 28 19:03:14.141374 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.141352    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hx2vn\""
Jul 28 19:03:14.141374 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.141369    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:03:14.209177 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.209145    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mj6fw"
Jul 28 19:03:14.400581 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.400554    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mj6fw"]
Jul 28 19:03:14.404928 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:14.404883    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa9a9e1_99e7_4bd1_9745_9231f6f83ba5.slice/crio-f8626723da20b7bd4678af985c838680fe5d43b214f1328b5e82d7a9b3e8734f WatchSource:0}: Error finding container f8626723da20b7bd4678af985c838680fe5d43b214f1328b5e82d7a9b3e8734f: Status 404 returned error can't find the container with id f8626723da20b7bd4678af985c838680fe5d43b214f1328b5e82d7a9b3e8734f
Jul 28 19:03:14.428483 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.428289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mj6fw" event={"ID":"daa9a9e1-99e7-4bd1-9745-9231f6f83ba5","Type":"ContainerStarted","Data":"f8626723da20b7bd4678af985c838680fe5d43b214f1328b5e82d7a9b3e8734f"}
Jul 28 19:03:14.431424 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.431395    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6e8d670-5c7a-4876-8249-85f13a045bab" containerID="d25392d2cb5f99b13ec2769007633178b7a6a3b852498d4737dfbe6e5ce1fb01" exitCode=0
Jul 28 19:03:14.431523 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.431430    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8lhnl" event={"ID":"b6e8d670-5c7a-4876-8249-85f13a045bab","Type":"ContainerDied","Data":"d25392d2cb5f99b13ec2769007633178b7a6a3b852498d4737dfbe6e5ce1fb01"}
Jul 28 19:03:14.603295 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:14.603205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:14.603411 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:14.603362    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:14.603452 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:14.603418    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls podName:1cf35915-6763-44f1-8d21-5ef270aad4f8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:15.603400539 +0000 UTC m=+37.065008228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls") pod "dns-default-smkjm" (UID: "1cf35915-6763-44f1-8d21-5ef270aad4f8") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:15.437890 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:15.437857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8lhnl" event={"ID":"b6e8d670-5c7a-4876-8249-85f13a045bab","Type":"ContainerStarted","Data":"d82ab2bc3495c25ff1bef9f743de31ba9587377fb6cca81969738c93c2f694a2"}
Jul 28 19:03:15.463112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:15.462846    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8lhnl" podStartSLOduration=7.751852614 podStartE2EDuration="36.462825951s" totalImagesPullingTime="28.710973337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:15.461531232 +0000 UTC m=+36.923138936" watchObservedRunningTime="2026-07-28 19:03:15.462825951 +0000 UTC m=+36.924433656"
Jul 28 19:03:15.584408 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:15.584353    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:15.612238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:15.612211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:15.612414 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:15.612356    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:15.612473 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:15.612416    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls podName:1cf35915-6763-44f1-8d21-5ef270aad4f8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:17.612400836 +0000 UTC m=+39.074008517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls") pod "dns-default-smkjm" (UID: "1cf35915-6763-44f1-8d21-5ef270aad4f8") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:15.684550 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:15.684516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:17.626750 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:17.626722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:17.627060 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:17.626861    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:17.627060 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:17.626944    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls podName:1cf35915-6763-44f1-8d21-5ef270aad4f8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:21.626906921 +0000 UTC m=+43.088514606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls") pod "dns-default-smkjm" (UID: "1cf35915-6763-44f1-8d21-5ef270aad4f8") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:18.446609 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:18.446424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mj6fw" event={"ID":"daa9a9e1-99e7-4bd1-9745-9231f6f83ba5","Type":"ContainerStarted","Data":"b29dd9ddff7533ca1da65de060c5db2902063855d9d7a99c831e2b7774460162"}
Jul 28 19:03:18.462914 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:18.462866    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mj6fw" podStartSLOduration=2.355646248 podStartE2EDuration="5.462847408s" totalImagesPullingTime="3.10720116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:18.461738688 +0000 UTC m=+39.923346391" watchObservedRunningTime="2026-07-28 19:03:18.462847408 +0000 UTC m=+39.924455097"
Jul 28 19:03:19.031183 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:19.031141    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:19.091197 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:19.091164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:21.653727 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:21.653688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:21.654116 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:21.653798    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:21.654116 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:21.653853    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls podName:1cf35915-6763-44f1-8d21-5ef270aad4f8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:29.653837302 +0000 UTC m=+51.115445003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls") pod "dns-default-smkjm" (UID: "1cf35915-6763-44f1-8d21-5ef270aad4f8") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:23.993958 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:23.993911    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"]
Jul 28 19:03:24.031666 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.031628    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"]
Jul 28 19:03:24.031809 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.031723    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:24.034522 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.034495    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 19:03:24.037066 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.037051    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 19:03:24.037375 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.037346    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 19:03:24.037375 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.037355    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pvlmv\""
Jul 28 19:03:24.037560 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.037492    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 19:03:24.108726 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.108693    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bnnpr"]
Jul 28 19:03:24.127314 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.127286    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.136461 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.135315    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cdcb77f48-xv9s2"]
Jul 28 19:03:24.156460 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.156436    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 19:03:24.156768 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.156747    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pcrdr\""
Jul 28 19:03:24.156864 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.156747    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 19:03:24.157026 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.157003    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 19:03:24.157026 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.157003    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 19:03:24.157432 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.157419    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.157495 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.157477    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4"]
Jul 28 19:03:24.161357 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.161342    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jcrsr\""
Jul 28 19:03:24.161493 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.161478    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 19:03:24.163670 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.163645    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 19:03:24.173013 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.172986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:24.173115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.173020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfln2\" (UniqueName: \"kubernetes.io/projected/141354af-c817-4cf9-b876-6e0e4852afd5-kube-api-access-xfln2\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:24.173115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.173095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/141354af-c817-4cf9-b876-6e0e4852afd5-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:24.174505 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.174484    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 19:03:24.177254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.177231    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bnnpr"]
Jul 28 19:03:24.177365 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.177282    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4"]
Jul 28 19:03:24.177365 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.177304    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cdcb77f48-xv9s2"]
Jul 28 19:03:24.177365 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.177309    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 19:03:24.177365 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.177323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4"
Jul 28 19:03:24.182590 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.182559    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 19:03:24.182590 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.182579    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2vq6n\""
Jul 28 19:03:24.182781 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.182567    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:03:24.207079 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.207051    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 19:03:24.274219 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274131    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-service-ca-bundle\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.274219 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-certificates\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.274219 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-bound-sa-token\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.274219 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-tmp\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.274476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-serving-cert\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.274476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274339    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-installation-pull-secrets\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.274476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-image-registry-private-configuration\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.274476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.274476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-trusted-ca\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.274476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-ca-trust-extracted\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.274476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5xlm\" (UniqueName: \"kubernetes.io/projected/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-kube-api-access-c5xlm\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.274753 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8cf9\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-kube-api-access-k8cf9\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.274753 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-snapshots\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.274753 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:24.274753 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfln2\" (UniqueName: \"kubernetes.io/projected/141354af-c817-4cf9-b876-6e0e4852afd5-kube-api-access-xfln2\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:24.274753 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2t66\" (UniqueName: \"kubernetes.io/projected/163fedcd-2a59-4baf-960b-064da188725e-kube-api-access-k2t66\") pod \"volume-data-source-validator-549c6d54cf-6ctm4\" (UID: \"163fedcd-2a59-4baf-960b-064da188725e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4"
Jul 28 19:03:24.274753 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/141354af-c817-4cf9-b876-6e0e4852afd5-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:24.274753 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.274651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.274753 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:24.274691    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:24.275046 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:24.274767    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls podName:141354af-c817-4cf9-b876-6e0e4852afd5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:24.774747024 +0000 UTC m=+46.236354724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pltw4" (UID: "141354af-c817-4cf9-b876-6e0e4852afd5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:24.275361 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.275345    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/141354af-c817-4cf9-b876-6e0e4852afd5-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:24.279688 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.279664    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:24.287132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.287113    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfln2\" (UniqueName: \"kubernetes.io/projected/141354af-c817-4cf9-b876-6e0e4852afd5-kube-api-access-xfln2\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:24.345399 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.345361    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:24.375024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.374999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2t66\" (UniqueName: \"kubernetes.io/projected/163fedcd-2a59-4baf-960b-064da188725e-kube-api-access-k2t66\") pod \"volume-data-source-validator-549c6d54cf-6ctm4\" (UID: \"163fedcd-2a59-4baf-960b-064da188725e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4"
Jul 28 19:03:24.375130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.375220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-service-ca-bundle\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.375263 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-certificates\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.375316 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-bound-sa-token\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.375316 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-tmp\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.375414 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-serving-cert\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.375414 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-installation-pull-secrets\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.375414 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-image-registry-private-configuration\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.375414 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375409    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.375614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-trusted-ca\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.375614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-ca-trust-extracted\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.375614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5xlm\" (UniqueName: \"kubernetes.io/projected/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-kube-api-access-c5xlm\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.375614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8cf9\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-kube-api-access-k8cf9\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.375614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375558    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-snapshots\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.375614 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:24.375608    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:03:24.375906 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:24.375626    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cdcb77f48-xv9s2: secret "image-registry-tls" not found
Jul 28 19:03:24.375906 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-tmp\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.375906 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:24.375683    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls podName:fc0dd43a-2a93-44f1-915f-0cb98f3ebce5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:24.875664066 +0000 UTC m=+46.337271748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls") pod "image-registry-5cdcb77f48-xv9s2" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5") : secret "image-registry-tls" not found
Jul 28 19:03:24.375906 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-service-ca-bundle\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.375906 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375888    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-certificates\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.376188 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375990    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.376188 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.375994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-ca-trust-extracted\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.376188 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.376178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-snapshots\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.376664 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.376598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-trusted-ca\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.377829 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.377804    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-serving-cert\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.378186 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.378167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-image-registry-private-configuration\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.378484 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.378468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-installation-pull-secrets\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.385308 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.385291    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8cf9\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-kube-api-access-k8cf9\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.385426 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.385406    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-bound-sa-token\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.385992 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.385971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2t66\" (UniqueName: \"kubernetes.io/projected/163fedcd-2a59-4baf-960b-064da188725e-kube-api-access-k2t66\") pod \"volume-data-source-validator-549c6d54cf-6ctm4\" (UID: \"163fedcd-2a59-4baf-960b-064da188725e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4"
Jul 28 19:03:24.386081 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.386066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5xlm\" (UniqueName: \"kubernetes.io/projected/8e7a991a-d67b-44a5-9be4-ad06be8c9d1e-kube-api-access-c5xlm\") pod \"insights-operator-5885fdb759-bnnpr\" (UID: \"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e\") " pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.426090 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.426059    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:24.438980 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.438964    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-bnnpr"
Jul 28 19:03:24.486297 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.486270    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4"
Jul 28 19:03:24.605507 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.605458    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bnnpr"]
Jul 28 19:03:24.609547 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:24.609519    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7a991a_d67b_44a5_9be4_ad06be8c9d1e.slice/crio-8187af28977e2cb47fd1960d711b9e7adf2df7a9a394c6667863fba0c990c6fe WatchSource:0}: Error finding container 8187af28977e2cb47fd1960d711b9e7adf2df7a9a394c6667863fba0c990c6fe: Status 404 returned error can't find the container with id 8187af28977e2cb47fd1960d711b9e7adf2df7a9a394c6667863fba0c990c6fe
Jul 28 19:03:24.648493 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.648465    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4"]
Jul 28 19:03:24.651786 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:24.651757    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163fedcd_2a59_4baf_960b_064da188725e.slice/crio-866a195102c1f9fa04cdf804e5d2e6cff92716fc05bbea36148e2f548f4f9fe8 WatchSource:0}: Error finding container 866a195102c1f9fa04cdf804e5d2e6cff92716fc05bbea36148e2f548f4f9fe8: Status 404 returned error can't find the container with id 866a195102c1f9fa04cdf804e5d2e6cff92716fc05bbea36148e2f548f4f9fe8
Jul 28 19:03:24.778193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.778159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:24.778348 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:24.778307    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:24.778387 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:24.778373    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls podName:141354af-c817-4cf9-b876-6e0e4852afd5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:25.778358192 +0000 UTC m=+47.239965873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pltw4" (UID: "141354af-c817-4cf9-b876-6e0e4852afd5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:24.879427 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:24.879399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:24.879589 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:24.879525    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:03:24.879589 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:24.879537    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cdcb77f48-xv9s2: secret "image-registry-tls" not found
Jul 28 19:03:24.879589 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:24.879581    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls podName:fc0dd43a-2a93-44f1-915f-0cb98f3ebce5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:25.879569282 +0000 UTC m=+47.341176963 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls") pod "image-registry-5cdcb77f48-xv9s2" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5") : secret "image-registry-tls" not found
Jul 28 19:03:25.463752 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:25.463710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bnnpr" event={"ID":"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e","Type":"ContainerStarted","Data":"8187af28977e2cb47fd1960d711b9e7adf2df7a9a394c6667863fba0c990c6fe"}
Jul 28 19:03:25.464735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:25.464707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4" event={"ID":"163fedcd-2a59-4baf-960b-064da188725e","Type":"ContainerStarted","Data":"866a195102c1f9fa04cdf804e5d2e6cff92716fc05bbea36148e2f548f4f9fe8"}
Jul 28 19:03:25.786965 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:25.786792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:25.787120 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:25.786986    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:25.787120 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:25.787068    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls podName:141354af-c817-4cf9-b876-6e0e4852afd5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:27.787045381 +0000 UTC m=+49.248653084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pltw4" (UID: "141354af-c817-4cf9-b876-6e0e4852afd5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:25.888386 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:25.888303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:25.888565 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:25.888481    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:03:25.888565 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:25.888504    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cdcb77f48-xv9s2: secret "image-registry-tls" not found
Jul 28 19:03:25.888673 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:25.888570    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls podName:fc0dd43a-2a93-44f1-915f-0cb98f3ebce5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:27.888549813 +0000 UTC m=+49.350157516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls") pod "image-registry-5cdcb77f48-xv9s2" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5") : secret "image-registry-tls" not found
Jul 28 19:03:27.471826 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:27.471739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4" event={"ID":"163fedcd-2a59-4baf-960b-064da188725e","Type":"ContainerStarted","Data":"040f4d22471933efd4d1b7dcd36ad275fde792565ad465d10407f4aafe8f4f5b"}
Jul 28 19:03:27.473350 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:27.473325    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bnnpr" event={"ID":"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e","Type":"ContainerStarted","Data":"acf04e669ccae04789e6603d18d7db5b2064027602fabdea38ec1e4c2227f97c"}
Jul 28 19:03:27.491655 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:27.491602    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6ctm4" podStartSLOduration=1.782653915 podStartE2EDuration="3.491586954s" totalImagesPullingTime="1.708933039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:27.489691943 +0000 UTC m=+48.951299645" watchObservedRunningTime="2026-07-28 19:03:27.491586954 +0000 UTC m=+48.953194694"
Jul 28 19:03:27.510391 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:27.510331    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-bnnpr" podStartSLOduration=1.032359862 podStartE2EDuration="3.510311003s" totalImagesPullingTime="2.477951141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:27.510063491 +0000 UTC m=+48.971671196" watchObservedRunningTime="2026-07-28 19:03:27.510311003 +0000 UTC m=+48.971918707"
Jul 28 19:03:27.632259 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:27.632222    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:27.804872 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:27.804769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:27.805050 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:27.804967    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:27.805050 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:27.805034    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls podName:141354af-c817-4cf9-b876-6e0e4852afd5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:31.805018561 +0000 UTC m=+53.266626242 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pltw4" (UID: "141354af-c817-4cf9-b876-6e0e4852afd5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:27.906129 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:27.906091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:27.906284 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:27.906243    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:03:27.906321 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:27.906287    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cdcb77f48-xv9s2: secret "image-registry-tls" not found
Jul 28 19:03:27.906356 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:27.906344    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls podName:fc0dd43a-2a93-44f1-915f-0cb98f3ebce5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:31.906325322 +0000 UTC m=+53.367933019 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls") pod "image-registry-5cdcb77f48-xv9s2" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5") : secret "image-registry-tls" not found
Jul 28 19:03:28.194154 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.194115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:28.492881 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.492805    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w55h5"]
Jul 28 19:03:28.514095 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.514066    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w55h5"]
Jul 28 19:03:28.514231 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.514182    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.517990 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.517969    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 19:03:28.518134 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.518006    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-26hct\""
Jul 28 19:03:28.518909 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.518893    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 19:03:28.557473 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.557436    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:28.594225 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.594196    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj"]
Jul 28 19:03:28.612376 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.612349    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.612525 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.612408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq2jh\" (UniqueName: \"kubernetes.io/projected/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-api-access-wq2jh\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.612525 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.612491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.612627 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.612529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-crio-socket\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.612724 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.612702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-data-volume\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.616172 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.616141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj"]
Jul 28 19:03:28.616318 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.616278    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj"
Jul 28 19:03:28.619214 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.619196    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 28 19:03:28.619214 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.619206    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-c28xf\""
Jul 28 19:03:28.619472 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.619460    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 28 19:03:28.714043 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.714013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.714043 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.714047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-crio-socket\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.714303 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.714095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-data-volume\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.714303 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:28.714199    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:28.714303 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.714211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.714303 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.714258    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-crio-socket\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.714303 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:28.714274    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:29.214252208 +0000 UTC m=+50.675859909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:28.714303 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:28.714294    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:28.714513 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.714328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcc72\" (UniqueName: \"kubernetes.io/projected/e891d093-8beb-46f1-9494-087e1f667296-kube-api-access-lcc72\") pod \"migrator-6865c9967-6z2zj\" (UID: \"e891d093-8beb-46f1-9494-087e1f667296\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj"
Jul 28 19:03:28.714513 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:28.714353    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:29.214337961 +0000 UTC m=+50.675945641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:28.714513 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.714379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2jh\" (UniqueName: \"kubernetes.io/projected/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-api-access-wq2jh\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.714513 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.714382    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-data-volume\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.724944 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.724899    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq2jh\" (UniqueName: \"kubernetes.io/projected/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-api-access-wq2jh\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:28.815477 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.815393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcc72\" (UniqueName: \"kubernetes.io/projected/e891d093-8beb-46f1-9494-087e1f667296-kube-api-access-lcc72\") pod \"migrator-6865c9967-6z2zj\" (UID: \"e891d093-8beb-46f1-9494-087e1f667296\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj"
Jul 28 19:03:28.824098 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.824067    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcc72\" (UniqueName: \"kubernetes.io/projected/e891d093-8beb-46f1-9494-087e1f667296-kube-api-access-lcc72\") pod \"migrator-6865c9967-6z2zj\" (UID: \"e891d093-8beb-46f1-9494-087e1f667296\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj"
Jul 28 19:03:28.925333 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:28.925292    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj"
Jul 28 19:03:29.075607 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:29.075452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj"]
Jul 28 19:03:29.077931 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:29.077886    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode891d093_8beb_46f1_9494_087e1f667296.slice/crio-287a58abc7bb1981e1b67e771be96aac24d56030992d049b9933a59c7c05b95f WatchSource:0}: Error finding container 287a58abc7bb1981e1b67e771be96aac24d56030992d049b9933a59c7c05b95f: Status 404 returned error can't find the container with id 287a58abc7bb1981e1b67e771be96aac24d56030992d049b9933a59c7c05b95f
Jul 28 19:03:29.218337 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:29.218297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:29.218557 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:29.218377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:29.218557 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:29.218439    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:29.218557 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:29.218505    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:29.218557 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:29.218519    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:30.218500557 +0000 UTC m=+51.680108256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:29.218557 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:29.218546    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:30.218535266 +0000 UTC m=+51.680142947 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:29.406222 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:29.406143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:29.479355 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:29.479321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj" event={"ID":"e891d093-8beb-46f1-9494-087e1f667296","Type":"ContainerStarted","Data":"287a58abc7bb1981e1b67e771be96aac24d56030992d049b9933a59c7c05b95f"}
Jul 28 19:03:29.517292 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:29.517264    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:29.710650 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:29.710564    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:29.722238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:29.722211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:29.722387 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:29.722365    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:29.722446 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:29.722436    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls podName:1cf35915-6763-44f1-8d21-5ef270aad4f8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:45.722416089 +0000 UTC m=+67.184023772 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls") pod "dns-default-smkjm" (UID: "1cf35915-6763-44f1-8d21-5ef270aad4f8") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:29.809846 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:29.809801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:29.994294 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:29.994213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:30.048315 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:30.048279    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:30.226280 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:30.226241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:30.226418 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:30.226355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:30.226418 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:30.226401    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:30.226506 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:30.226458    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:30.226506 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:30.226472    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:32.226456416 +0000 UTC m=+53.688064099 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:30.226588 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:30.226510    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:32.226493816 +0000 UTC m=+53.688101516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:30.465307 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:30.465283    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ckcmv_ed3429b6-f5f2-48ab-a368-28543e3860ad/dns-node-resolver/0.log"
Jul 28 19:03:31.464275 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:31.464248    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9k5kh_645352cb-de5d-4b0b-88dc-670476f0deff/node-ca/0.log"
Jul 28 19:03:31.626612 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:31.626581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:31.693209 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:31.693174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:31.838663 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:31.838634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:31.838806 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:31.838788    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:31.838857 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:31.838848    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls podName:141354af-c817-4cf9-b876-6e0e4852afd5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:39.838832656 +0000 UTC m=+61.300440337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pltw4" (UID: "141354af-c817-4cf9-b876-6e0e4852afd5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:31.939048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:31.939006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:31.939209 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:31.939151    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:03:31.939209 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:31.939173    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cdcb77f48-xv9s2: secret "image-registry-tls" not found
Jul 28 19:03:31.939288 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:31.939228    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls podName:fc0dd43a-2a93-44f1-915f-0cb98f3ebce5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:39.939213381 +0000 UTC m=+61.400821064 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls") pod "image-registry-5cdcb77f48-xv9s2" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5") : secret "image-registry-tls" not found
Jul 28 19:03:32.241796 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:32.241763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:32.241984 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:32.241840    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:32.241984 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:32.241912    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:32.241984 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:32.241967    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:32.242091 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:32.242001    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:36.241984794 +0000 UTC m=+57.703592475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:32.242091 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:32.242015    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:36.242009579 +0000 UTC m=+57.703617259 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:32.466554 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:32.466528    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mj6fw_daa9a9e1-99e7-4bd1-9745-9231f6f83ba5/serve-healthcheck-canary/0.log"
Jul 28 19:03:32.494185 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:32.494106    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj" event={"ID":"e891d093-8beb-46f1-9494-087e1f667296","Type":"ContainerStarted","Data":"f341214bfa0d280456f34c16924d624dd91c1f784d58e1df14358c03e1e749b5"}
Jul 28 19:03:32.494309 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:32.494183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj" event={"ID":"e891d093-8beb-46f1-9494-087e1f667296","Type":"ContainerStarted","Data":"3e4a38c68bcae722b91eef95525f0ad2542953ae41a131280e2e1191e3df8758"}
Jul 28 19:03:32.515156 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:32.515107    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6z2zj" podStartSLOduration=1.899404653 podStartE2EDuration="4.515092684s" totalImagesPullingTime="2.615688031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:32.513578985 +0000 UTC m=+53.975186684" watchObservedRunningTime="2026-07-28 19:03:32.515092684 +0000 UTC m=+53.976700387"
Jul 28 19:03:32.588126 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:32.588099    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:35.466724 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:35.466690    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:35.674503 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:35.674462    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:35.825209 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:35.825182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:35.919735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:35.919701    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:36.020731 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:36.020702    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:36.176330 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:36.176251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:36.276481 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:36.276441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:36.276645 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:36.276529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:36.276645 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:36.276562    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:36.276645 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:36.276643    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:44.276625099 +0000 UTC m=+65.738232781 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:36.276767 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:36.276647    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:36.276767 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:36.276700    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:44.276688007 +0000 UTC m=+65.738295690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:36.959706 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:36.959675    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:37.259782 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:37.259704    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:37.577890 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:37.577858    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:38.134237 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:38.134212    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:38.233633 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:38.233604    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:38.370682 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:38.370654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:38.774450 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:38.774418    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:39.163868 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:39.163831    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:39.296837 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:39.296793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:39.422948 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:39.422863    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-62f22"
Jul 28 19:03:39.566839 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:39.566805    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:39.761459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:39.761381    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:39.909807 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:39.909772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:39.909988 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:39.909932    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:39.910037 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:39.909998    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls podName:141354af-c817-4cf9-b876-6e0e4852afd5 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:55.909982271 +0000 UTC m=+77.371589953 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pltw4" (UID: "141354af-c817-4cf9-b876-6e0e4852afd5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:03:40.010852 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.010811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:40.013359 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.013316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls\") pod \"image-registry-5cdcb77f48-xv9s2\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") " pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:40.067853 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.067821    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jcrsr\""
Jul 28 19:03:40.076017 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.075976    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:40.164340 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.164314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:40.204835 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.204797    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cdcb77f48-xv9s2"]
Jul 28 19:03:40.208642 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:40.208617    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0dd43a_2a93_44f1_915f_0cb98f3ebce5.slice/crio-c0461964defa65b8cddcc21065078904d1a064b1484acba0a589e43100c166ce WatchSource:0}: Error finding container c0461964defa65b8cddcc21065078904d1a064b1484acba0a589e43100c166ce: Status 404 returned error can't find the container with id c0461964defa65b8cddcc21065078904d1a064b1484acba0a589e43100c166ce
Jul 28 19:03:40.264500 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.264416    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:40.366285 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.366253    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:40.517681 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.517599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2" event={"ID":"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5","Type":"ContainerStarted","Data":"488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09"}
Jul 28 19:03:40.517681 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.517642    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2" event={"ID":"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5","Type":"ContainerStarted","Data":"c0461964defa65b8cddcc21065078904d1a064b1484acba0a589e43100c166ce"}
Jul 28 19:03:40.517681 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.517670    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:03:40.538730 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.538584    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2" podStartSLOduration=16.538564832 podStartE2EDuration="16.538564832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:40.538004324 +0000 UTC m=+61.999612027" watchObservedRunningTime="2026-07-28 19:03:40.538564832 +0000 UTC m=+62.000172530"
Jul 28 19:03:40.779581 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:40.779464    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:41.570502 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:41.570473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:41.692174 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:41.692139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:41.794592 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:41.794563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:42.187041 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:42.187011    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:42.244751 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:42.244719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:42.970847 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:42.970797    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:43.365171 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:43.365135    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:43.571128 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:43.571094    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:43.841277 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:43.841242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:43.976682 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:43.976650    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:44.348019 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.347986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:44.348209 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.348061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:44.348209 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:44.348172    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:44.348331 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:03:44.348225    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls podName:0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:04:00.348211903 +0000 UTC m=+81.809819584 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w55h5" (UID: "0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:44.348565 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.348542    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:03:44.566077 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.566038    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:44.751221 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.751123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:44.751221 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.751179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:44.753983 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.753960    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:03:44.755093 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.755081    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:03:44.764499 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.764474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c6fda6c-c699-4ce6-a78a-209994e2db79-original-pull-secret\") pod \"global-pull-secret-syncer-wl5w9\" (UID: \"7c6fda6c-c699-4ce6-a78a-209994e2db79\") " pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:44.764499 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.764492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88ae7e56-364a-4f77-862a-2065daefbcfc-metrics-certs\") pod \"network-metrics-daemon-9xt7v\" (UID: \"88ae7e56-364a-4f77-862a-2065daefbcfc\") " pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:44.852539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.852502    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwps\" (UniqueName: \"kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps\") pod \"network-check-target-29nx8\" (UID: \"45d06a99-391d-46ef-8ae4-c6858cfa6511\") " pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:44.855190 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.855166    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:03:44.865801 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.865779    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:03:44.876562 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.876542    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbwps\" (UniqueName: \"kubernetes.io/projected/45d06a99-391d-46ef-8ae4-c6858cfa6511-kube-api-access-xbwps\") pod \"network-check-target-29nx8\" (UID: \"45d06a99-391d-46ef-8ae4-c6858cfa6511\") " pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:44.994930 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:44.994894    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:45.051298 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.051214    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-snpbv\""
Jul 28 19:03:45.055659 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.055639    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hx2vn\""
Jul 28 19:03:45.058874 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.058861    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xt7v"
Jul 28 19:03:45.058977 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.058893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl5w9"
Jul 28 19:03:45.063965 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.063903    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:45.210970 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.210943    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-29nx8"]
Jul 28 19:03:45.214406 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:45.214375    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45d06a99_391d_46ef_8ae4_c6858cfa6511.slice/crio-96531c3529895ee3cd1bc34e744bd60b3293d1fbea750b58587d4ec6d5abed06 WatchSource:0}: Error finding container 96531c3529895ee3cd1bc34e744bd60b3293d1fbea750b58587d4ec6d5abed06: Status 404 returned error can't find the container with id 96531c3529895ee3cd1bc34e744bd60b3293d1fbea750b58587d4ec6d5abed06
Jul 28 19:03:45.225306 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.225284    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wl5w9"]
Jul 28 19:03:45.228637 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:45.228611    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6fda6c_c699_4ce6_a78a_209994e2db79.slice/crio-a1832a748ab080d2561ff2e1f966feb188abf0ef8ac765165eba938931b87b21 WatchSource:0}: Error finding container a1832a748ab080d2561ff2e1f966feb188abf0ef8ac765165eba938931b87b21: Status 404 returned error can't find the container with id a1832a748ab080d2561ff2e1f966feb188abf0ef8ac765165eba938931b87b21
Jul 28 19:03:45.242265 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.242247    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9xt7v"]
Jul 28 19:03:45.244278 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:45.244259    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88ae7e56_364a_4f77_862a_2065daefbcfc.slice/crio-5c032a973e327c999e21f97a0288c886359e6583e87f82e01171ac48a93ecc3c WatchSource:0}: Error finding container 5c032a973e327c999e21f97a0288c886359e6583e87f82e01171ac48a93ecc3c: Status 404 returned error can't find the container with id 5c032a973e327c999e21f97a0288c886359e6583e87f82e01171ac48a93ecc3c
Jul 28 19:03:45.374806 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.374777    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:45.535207 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.535171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wl5w9" event={"ID":"7c6fda6c-c699-4ce6-a78a-209994e2db79","Type":"ContainerStarted","Data":"a1832a748ab080d2561ff2e1f966feb188abf0ef8ac765165eba938931b87b21"}
Jul 28 19:03:45.536215 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.536193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9xt7v" event={"ID":"88ae7e56-364a-4f77-862a-2065daefbcfc","Type":"ContainerStarted","Data":"5c032a973e327c999e21f97a0288c886359e6583e87f82e01171ac48a93ecc3c"}
Jul 28 19:03:45.537316 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.537297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-29nx8" event={"ID":"45d06a99-391d-46ef-8ae4-c6858cfa6511","Type":"ContainerStarted","Data":"96531c3529895ee3cd1bc34e744bd60b3293d1fbea750b58587d4ec6d5abed06"}
Jul 28 19:03:45.759775 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.759691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:45.761473 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.761415    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:45.762579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.762557    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1cf35915-6763-44f1-8d21-5ef270aad4f8-metrics-tls\") pod \"dns-default-smkjm\" (UID: \"1cf35915-6763-44f1-8d21-5ef270aad4f8\") " pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:45.965308 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.965275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:45.995829 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:45.995799    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d4lpd\""
Jul 28 19:03:46.003258 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:46.003236    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:46.167877 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:46.167810    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:46.247265 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:46.247239    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-smkjm"]
Jul 28 19:03:46.253122 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:46.253097    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf35915_6763_44f1_8d21_5ef270aad4f8.slice/crio-058edf21096ef07c64b970e3192f6b00187077008c382ce3bd09476bf14bca68 WatchSource:0}: Error finding container 058edf21096ef07c64b970e3192f6b00187077008c382ce3bd09476bf14bca68: Status 404 returned error can't find the container with id 058edf21096ef07c64b970e3192f6b00187077008c382ce3bd09476bf14bca68
Jul 28 19:03:46.543702 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:46.543619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-smkjm" event={"ID":"1cf35915-6763-44f1-8d21-5ef270aad4f8","Type":"ContainerStarted","Data":"058edf21096ef07c64b970e3192f6b00187077008c382ce3bd09476bf14bca68"}
Jul 28 19:03:46.545967 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:46.545939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9xt7v" event={"ID":"88ae7e56-364a-4f77-862a-2065daefbcfc","Type":"ContainerStarted","Data":"39904e5bb7bad058ca2f30ba08df4ff35e08b61a28f5d974febb3ae165d6c4bf"}
Jul 28 19:03:46.546108 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:46.545995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9xt7v" event={"ID":"88ae7e56-364a-4f77-862a-2065daefbcfc","Type":"ContainerStarted","Data":"c47bc2c60767a2e31e843982625da571aec895b291306a4e7626a8dad1acded8"}
Jul 28 19:03:46.576672 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:46.576617    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9xt7v" podStartSLOduration=66.689426775 podStartE2EDuration="1m7.576596102s" totalImagesPullingTime="887.169327ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:46.568701205 +0000 UTC m=+68.030308924" watchObservedRunningTime="2026-07-28 19:03:46.576596102 +0000 UTC m=+68.038203805"
Jul 28 19:03:46.949882 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:46.949840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:47.020266 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:47.020230    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:50.563438 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:50.563349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-29nx8" event={"ID":"45d06a99-391d-46ef-8ae4-c6858cfa6511","Type":"ContainerStarted","Data":"59c27cec01a28f8fc6845e282ba7daf48c8d3757fb705cd94fc0918742f75f22"}
Jul 28 19:03:50.563438 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:50.563406    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:03:50.564817 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:50.564789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wl5w9" event={"ID":"7c6fda6c-c699-4ce6-a78a-209994e2db79","Type":"ContainerStarted","Data":"c33a9ea5ed63df07433d4a127cabf119d2ddab24bdab5712323b3aca83d2edfc"}
Jul 28 19:03:50.566412 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:50.566390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-smkjm" event={"ID":"1cf35915-6763-44f1-8d21-5ef270aad4f8","Type":"ContainerStarted","Data":"164fd7fc98feac04cca180d312f5e71cc9d5f4262aeebbdf5db5fec51904533d"}
Jul 28 19:03:50.566521 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:50.566432    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-smkjm" event={"ID":"1cf35915-6763-44f1-8d21-5ef270aad4f8","Type":"ContainerStarted","Data":"d0b24ae70ed143c81ce783571480005d356561b20a162a6820bd49231155d1cb"}
Jul 28 19:03:50.566577 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:50.566523    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-smkjm"
Jul 28 19:03:50.580333 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:50.580281    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-29nx8" podStartSLOduration=66.602501077 podStartE2EDuration="1m11.580266406s" totalImagesPullingTime="4.977765329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:50.57964451 +0000 UTC m=+72.041252227" watchObservedRunningTime="2026-07-28 19:03:50.580266406 +0000 UTC m=+72.041874109"
Jul 28 19:03:50.596287 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:50.596237    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wl5w9" podStartSLOduration=66.619078361 podStartE2EDuration="1m11.596222093s" totalImagesPullingTime="4.977143732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:50.595451855 +0000 UTC m=+72.057059573" watchObservedRunningTime="2026-07-28 19:03:50.596222093 +0000 UTC m=+72.057829796"
Jul 28 19:03:50.614298 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:50.613768    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-smkjm" podStartSLOduration=33.672779354 podStartE2EDuration="37.61375398s" totalImagesPullingTime="3.940974626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:50.613125358 +0000 UTC m=+72.074733060" watchObservedRunningTime="2026-07-28 19:03:50.61375398 +0000 UTC m=+72.075361684"
Jul 28 19:03:54.926389 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.926356    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"]
Jul 28 19:03:54.928397 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.928382    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:54.931033 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.931010    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 19:03:54.931145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.931076    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 19:03:54.931145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.931111    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 19:03:54.931301 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.931288    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 19:03:54.932187 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.932172    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 19:03:54.932294 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.932279    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 19:03:54.932353 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.932330    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 19:03:54.947693 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.947668    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"]
Jul 28 19:03:54.959533 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.959509    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cdcb77f48-xv9s2"]
Jul 28 19:03:54.989594 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:54.989556    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:55.038880 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.038845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-hub\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.038880 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.038881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.039090 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.039022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-ca\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.039090 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.039050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.039153 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.039090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/16b2cf59-1a42-4bba-94c0-ae6206599cf7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.039153 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.039122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m28f7\" (UniqueName: \"kubernetes.io/projected/16b2cf59-1a42-4bba-94c0-ae6206599cf7-kube-api-access-m28f7\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.105579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.105536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:55.140162 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.140133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-ca\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.140162 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.140168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.140417 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.140214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/16b2cf59-1a42-4bba-94c0-ae6206599cf7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.140417 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.140253    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m28f7\" (UniqueName: \"kubernetes.io/projected/16b2cf59-1a42-4bba-94c0-ae6206599cf7-kube-api-access-m28f7\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.140417 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.140284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-hub\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.140568 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.140484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.141135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.141082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/16b2cf59-1a42-4bba-94c0-ae6206599cf7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.143159 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.143126    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.143277 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.143194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-hub\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.143330 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.143299    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.143371 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.143333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/16b2cf59-1a42-4bba-94c0-ae6206599cf7-ca\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.165183 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.165152    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m28f7\" (UniqueName: \"kubernetes.io/projected/16b2cf59-1a42-4bba-94c0-ae6206599cf7-kube-api-access-m28f7\") pod \"cluster-proxy-proxy-agent-7f59b98467-rc9lc\" (UID: \"16b2cf59-1a42-4bba-94c0-ae6206599cf7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.208777 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.208698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:55.246361 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.246336    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"
Jul 28 19:03:55.308558 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.308516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:55.385589 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.385567    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc"]
Jul 28 19:03:55.388102 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:55.388072    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16b2cf59_1a42_4bba_94c0_ae6206599cf7.slice/crio-2aa3e12888eae425fe0a25e90cb142d9b529a20c0c1cb40ba7b5f85be926dd4d WatchSource:0}: Error finding container 2aa3e12888eae425fe0a25e90cb142d9b529a20c0c1cb40ba7b5f85be926dd4d: Status 404 returned error can't find the container with id 2aa3e12888eae425fe0a25e90cb142d9b529a20c0c1cb40ba7b5f85be926dd4d
Jul 28 19:03:55.396042 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.396020    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:55.461680 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.461599    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:55.585120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.585086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc" event={"ID":"16b2cf59-1a42-4bba-94c0-ae6206599cf7","Type":"ContainerStarted","Data":"2aa3e12888eae425fe0a25e90cb142d9b529a20c0c1cb40ba7b5f85be926dd4d"}
Jul 28 19:03:55.946878 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.946823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:55.949340 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:55.949322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/141354af-c817-4cf9-b876-6e0e4852afd5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pltw4\" (UID: \"141354af-c817-4cf9-b876-6e0e4852afd5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:56.143148 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:56.143118    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pvlmv\""
Jul 28 19:03:56.151515 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:56.151490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"
Jul 28 19:03:56.281028 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:56.280994    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4"]
Jul 28 19:03:56.283750 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:03:56.283722    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141354af_c817_4cf9_b876_6e0e4852afd5.slice/crio-8332758fddc23b9cfb8f0a3ebf87e02b5906ab32924afef217bd26c6e06879ee WatchSource:0}: Error finding container 8332758fddc23b9cfb8f0a3ebf87e02b5906ab32924afef217bd26c6e06879ee: Status 404 returned error can't find the container with id 8332758fddc23b9cfb8f0a3ebf87e02b5906ab32924afef217bd26c6e06879ee
Jul 28 19:03:56.589591 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:56.589559    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4" event={"ID":"141354af-c817-4cf9-b876-6e0e4852afd5","Type":"ContainerStarted","Data":"8332758fddc23b9cfb8f0a3ebf87e02b5906ab32924afef217bd26c6e06879ee"}
Jul 28 19:03:58.233496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:58.233407    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:58.333117 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:58.333081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:58.409632 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:58.409582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:58.598839 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:58.598803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4" event={"ID":"141354af-c817-4cf9-b876-6e0e4852afd5","Type":"ContainerStarted","Data":"5f18346666a39b4a96f65bb89ab61e4e8808a452076215336ac78463a75ef793"}
Jul 28 19:03:58.617038 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:58.616989    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pltw4" podStartSLOduration=34.01039061 podStartE2EDuration="35.616935309s" totalImagesPullingTime="1.606544699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:58.615973822 +0000 UTC m=+80.077581526" watchObservedRunningTime="2026-07-28 19:03:58.616935309 +0000 UTC m=+80.078543003"
Jul 28 19:03:58.646164 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:58.646115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:59.181577 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:59.181544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:59.291354 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:03:59.291243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:00.385342 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:00.385307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:04:00.387831 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:00.387806    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w55h5\" (UID: \"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8\") " pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:04:00.572369 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:00.572343    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-smkjm"
Jul 28 19:04:00.626727 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:00.626696    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-26hct\""
Jul 28 19:04:00.633967 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:00.633940    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w55h5"
Jul 28 19:04:00.784025 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:00.783990    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w55h5"]
Jul 28 19:04:01.843023 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:04:01.842947    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e2bd23e_e1a4_4d74_b7ca_065ef5c24fb8.slice/crio-8972cb2f4fe6f64bd10072959dfb6d892bee8aea57fc1e47466f144c60b982f9 WatchSource:0}: Error finding container 8972cb2f4fe6f64bd10072959dfb6d892bee8aea57fc1e47466f144c60b982f9: Status 404 returned error can't find the container with id 8972cb2f4fe6f64bd10072959dfb6d892bee8aea57fc1e47466f144c60b982f9
Jul 28 19:04:02.421473 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:02.421367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:02.526034 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:02.525950    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:02.618740 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:02.618153    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc" event={"ID":"16b2cf59-1a42-4bba-94c0-ae6206599cf7","Type":"ContainerStarted","Data":"8118abbb0ad0f702208b0ffb881488839320b604899e343144ad0617b1497dde"}
Jul 28 19:04:02.618740 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:02.618207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc" event={"ID":"16b2cf59-1a42-4bba-94c0-ae6206599cf7","Type":"ContainerStarted","Data":"a4ea10fd49d0ae9e72ce231a668eeb7489b92d88d557ef4da3c714ed3b95e332"}
Jul 28 19:04:02.618740 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:02.618225    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc" event={"ID":"16b2cf59-1a42-4bba-94c0-ae6206599cf7","Type":"ContainerStarted","Data":"8b1381af3e1ca1f5852b2fece4726a22d8ba2cf9574221ed37be80fdb946cce7"}
Jul 28 19:04:02.620504 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:02.620472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w55h5" event={"ID":"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8","Type":"ContainerStarted","Data":"8972cb2f4fe6f64bd10072959dfb6d892bee8aea57fc1e47466f144c60b982f9"}
Jul 28 19:04:02.639059 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:02.639003    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f59b98467-rc9lc" podStartSLOduration=2.123177635 podStartE2EDuration="8.638981472s" totalImagesPullingTime="6.515803837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:02.638421101 +0000 UTC m=+84.100028804" watchObservedRunningTime="2026-07-28 19:04:02.638981472 +0000 UTC m=+84.100589176"
Jul 28 19:04:03.104403 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:03.104364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:04.160243 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:04.160211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:04.226962 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:04.226893    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:04.966013 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:04.965983    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:04:05.165002 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:05.164961    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:05.265592 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:05.265511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:07.547756 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:07.547723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:07.640398 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:07.640369    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w55h5" event={"ID":"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8","Type":"ContainerStarted","Data":"16d90246f0888545e06c13186f6b481887705ec0c6d8f28d1a64ad956fd7d7d8"}
Jul 28 19:04:09.567268 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.567214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:09.821901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.821862    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:09.846436 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.846402    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wrrhb"]
Jul 28 19:04:09.857670 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.857642    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:09.860579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.860551    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 19:04:09.861563 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.861143    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 19:04:09.861563 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.861182    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 19:04:09.861563 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.861424    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 19:04:09.861893 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.861876    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kmxrs\""
Jul 28 19:04:09.921694 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.921661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:09.965266 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.965234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-root\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:09.965440 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.965273    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-sys\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:09.965440 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.965299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:09.965440 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.965340    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v6wn\" (UniqueName: \"kubernetes.io/projected/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-kube-api-access-8v6wn\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:09.965440 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.965368    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-textfile\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:09.965440 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.965410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:09.965440 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.965436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-wtmp\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:09.965674 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.965457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-metrics-client-ca\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:09.965674 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:09.965493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-tls\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.035077 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.035046    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:10.066153 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-root\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-sys\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v6wn\" (UniqueName: \"kubernetes.io/projected/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-kube-api-access-8v6wn\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066251    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-root\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-textfile\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066324    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-sys\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066364    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-wtmp\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-metrics-client-ca\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-tls\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-textfile\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066662    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-wtmp\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.066764 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:04:10.066715    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 19:04:10.066764 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:04:10.066759    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-tls podName:5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0 nodeName:}" failed. No retries permitted until 2026-07-28 19:04:10.566744598 +0000 UTC m=+92.028352282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-tls") pod "node-exporter-wrrhb" (UID: "5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0") : secret "node-exporter-tls" not found
Jul 28 19:04:10.067004 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.066820    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.067150 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.067087    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-metrics-client-ca\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.069246 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.069227    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.079040 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.078983    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v6wn\" (UniqueName: \"kubernetes.io/projected/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-kube-api-access-8v6wn\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.145025 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.144988    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:10.268077 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.268040    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:10.427030 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.426957    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:10.526438 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.526393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:10.570657 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.570631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-tls\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.573439 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.573410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0-node-exporter-tls\") pod \"node-exporter-wrrhb\" (UID: \"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0\") " pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.626962 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.626908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:10.769036 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.768944    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wrrhb"
Jul 28 19:04:10.836290 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.836255    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:10.964291 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:10.964254    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:11.060474 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:04:11.060430    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d98dbcb_d508_4b66_b2c2_ef2e2cd55eb0.slice/crio-02e86c7ca1007a8bf3ce5aabbe7246280537c32a41a986d3e49dd085ad05caf5 WatchSource:0}: Error finding container 02e86c7ca1007a8bf3ce5aabbe7246280537c32a41a986d3e49dd085ad05caf5: Status 404 returned error can't find the container with id 02e86c7ca1007a8bf3ce5aabbe7246280537c32a41a986d3e49dd085ad05caf5
Jul 28 19:04:11.260626 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:11.260597    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:11.661932 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:11.661895    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w55h5" event={"ID":"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8","Type":"ContainerStarted","Data":"cd917819d44a9627f1311179beeb4a3257947f8f849072f56203fdbfb7644168"}
Jul 28 19:04:11.663346 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:11.663318    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrrhb" event={"ID":"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0","Type":"ContainerStarted","Data":"02e86c7ca1007a8bf3ce5aabbe7246280537c32a41a986d3e49dd085ad05caf5"}
Jul 28 19:04:12.778771 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:12.778739    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:12.881266 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:12.881236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:13.178477 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:13.178442    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:13.287989 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:13.287957    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:13.673820 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:13.673787    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0" containerID="0baa16c38d55b3b7345dfdbbba63ce4c2d73a7c9ac00c5885af0ebf2a8646ba9" exitCode=0
Jul 28 19:04:13.674021 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:13.673829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrrhb" event={"ID":"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0","Type":"ContainerDied","Data":"0baa16c38d55b3b7345dfdbbba63ce4c2d73a7c9ac00c5885af0ebf2a8646ba9"}
Jul 28 19:04:13.680081 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:13.680060    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:13.818587 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:13.818551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:14.275217 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:14.274957    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:14.372765 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:14.372720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:14.618544 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:14.618494    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:15.248130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:15.248100    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:15.682208 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:15.682177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w55h5" event={"ID":"0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8","Type":"ContainerStarted","Data":"4d074cacbc521f27895bfa95730e91f1ba8e576dbe09054d0982fb4c6492f0d5"}
Jul 28 19:04:15.684174 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:15.684155    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrrhb" event={"ID":"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0","Type":"ContainerStarted","Data":"a7eb24d89e32daaa5bffb2e78cf0cba2f2d195c9dddf28dd6f02cb1611c3462a"}
Jul 28 19:04:15.684264 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:15.684190    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrrhb" event={"ID":"5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0","Type":"ContainerStarted","Data":"8aecb54c09d6e455846090f689f600253971dc8759dcaf08800573822682a24c"}
Jul 28 19:04:15.705284 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:15.705227    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w55h5" podStartSLOduration=34.897060191 podStartE2EDuration="47.70521315s" totalImagesPullingTime="12.808152959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:03:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:15.70419703 +0000 UTC m=+97.165804733" watchObservedRunningTime="2026-07-28 19:04:15.70521315 +0000 UTC m=+97.166820849"
Jul 28 19:04:15.725396 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:15.725344    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wrrhb" podStartSLOduration=4.940587393 podStartE2EDuration="6.725324543s" totalImagesPullingTime="1.703587323s" totalInitContainerRuntime="81.149827ms" isStatefulPod=true podCreationTimestamp="2026-07-28 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:15.724520538 +0000 UTC m=+97.186128243" watchObservedRunningTime="2026-07-28 19:04:15.725324543 +0000 UTC m=+97.186932298"
Jul 28 19:04:16.119623 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:16.119593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:16.937105 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:16.937067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:17.041062 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:17.041033    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:17.279148 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:17.279076    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:17.380960 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:17.380908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:19.978995 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:19.978905    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2" podUID="fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" containerName="registry" containerID="cri-o://488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09" gracePeriod=30
Jul 28 19:04:20.216872 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.216852    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:04:20.322433 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.322399    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-vhsn4"]
Jul 28 19:04:20.322724 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.322710    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" containerName="registry"
Jul 28 19:04:20.322776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.322726    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" containerName="registry"
Jul 28 19:04:20.322809 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.322775    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" containerName="registry"
Jul 28 19:04:20.324307 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.324293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-vhsn4"
Jul 28 19:04:20.327542 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.327524    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 19:04:20.328380 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.328365    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 19:04:20.328459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.328445    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rfdcf\""
Jul 28 19:04:20.338030 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.338004    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-vhsn4"]
Jul 28 19:04:20.346996 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.346974    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-ca-trust-extracted\") pod \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") "
Jul 28 19:04:20.347114 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.347013    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-installation-pull-secrets\") pod \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") "
Jul 28 19:04:20.347114 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.347040    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls\") pod \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") "
Jul 28 19:04:20.347114 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.347077    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-certificates\") pod \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") "
Jul 28 19:04:20.347267 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.347118    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-image-registry-private-configuration\") pod \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") "
Jul 28 19:04:20.347267 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.347159    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-trusted-ca\") pod \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") "
Jul 28 19:04:20.347267 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.347213    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8cf9\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-kube-api-access-k8cf9\") pod \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") "
Jul 28 19:04:20.347267 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.347237    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-bound-sa-token\") pod \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\" (UID: \"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5\") "
Jul 28 19:04:20.347751 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.347670    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:04:20.348225 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.347862    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:04:20.348609 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.348546    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:20.349937 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.349872    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:04:20.349937 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.349903    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:04:20.350116 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.350092    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:04:20.350191 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.350172    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:04:20.350457 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.350439    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-kube-api-access-k8cf9" (OuterVolumeSpecName: "kube-api-access-k8cf9") pod "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5"). InnerVolumeSpecName "kube-api-access-k8cf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:04:20.355813 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.355789    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" (UID: "fc0dd43a-2a93-44f1-915f-0cb98f3ebce5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:04:20.448585 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.448535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvfzg\" (UniqueName: \"kubernetes.io/projected/e684e3a5-5494-43a3-884e-655779d70d39-kube-api-access-dvfzg\") pod \"downloads-7c97dc4555-vhsn4\" (UID: \"e684e3a5-5494-43a3-884e-655779d70d39\") " pod="openshift-console/downloads-7c97dc4555-vhsn4"
Jul 28 19:04:20.448764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.448658    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8cf9\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-kube-api-access-k8cf9\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:04:20.448764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.448672    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-bound-sa-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:04:20.448764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.448681    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-ca-trust-extracted\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:04:20.448764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.448690    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-installation-pull-secrets\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:04:20.448764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.448699    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-tls\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:04:20.448764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.448707    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-registry-certificates\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:04:20.448764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.448716    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-image-registry-private-configuration\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:04:20.448764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.448724    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5-trusted-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:04:20.449815 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.449792    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:20.549445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.549418    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvfzg\" (UniqueName: \"kubernetes.io/projected/e684e3a5-5494-43a3-884e-655779d70d39-kube-api-access-dvfzg\") pod \"downloads-7c97dc4555-vhsn4\" (UID: \"e684e3a5-5494-43a3-884e-655779d70d39\") " pod="openshift-console/downloads-7c97dc4555-vhsn4"
Jul 28 19:04:20.558512 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.558482    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvfzg\" (UniqueName: \"kubernetes.io/projected/e684e3a5-5494-43a3-884e-655779d70d39-kube-api-access-dvfzg\") pod \"downloads-7c97dc4555-vhsn4\" (UID: \"e684e3a5-5494-43a3-884e-655779d70d39\") " pod="openshift-console/downloads-7c97dc4555-vhsn4"
Jul 28 19:04:20.618295 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.618228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:20.633159 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.633139    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-vhsn4"
Jul 28 19:04:20.703792 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.703759    2563 generic.go:356] "Generic (PLEG): container finished" podID="fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" containerID="488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09" exitCode=0
Jul 28 19:04:20.703997 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.703802    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2"
Jul 28 19:04:20.703997 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.703834    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2" event={"ID":"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5","Type":"ContainerDied","Data":"488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09"}
Jul 28 19:04:20.703997 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.703896    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cdcb77f48-xv9s2" event={"ID":"fc0dd43a-2a93-44f1-915f-0cb98f3ebce5","Type":"ContainerDied","Data":"c0461964defa65b8cddcc21065078904d1a064b1484acba0a589e43100c166ce"}
Jul 28 19:04:20.703997 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.703977    2563 scope.go:117] "RemoveContainer" containerID="488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09"
Jul 28 19:04:20.712742 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.712721    2563 scope.go:117] "RemoveContainer" containerID="488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09"
Jul 28 19:04:20.713041 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:04:20.713022    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09\": container with ID starting with 488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09 not found: ID does not exist" containerID="488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09"
Jul 28 19:04:20.713123 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.713048    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09"} err="failed to get container status \"488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09\": rpc error: code = NotFound desc = could not find container \"488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09\": container with ID starting with 488a94befb37ef28a1075c29eb8d977491dbc39d050462523dc5be5bd9f81f09 not found: ID does not exist"
Jul 28 19:04:20.725710 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.725683    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cdcb77f48-xv9s2"]
Jul 28 19:04:20.731010 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.730984    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cdcb77f48-xv9s2"]
Jul 28 19:04:20.770250 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.770211    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-vhsn4"]
Jul 28 19:04:20.774873 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:04:20.774847    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode684e3a5_5494_43a3_884e_655779d70d39.slice/crio-14288b5787536272fe8f860ad007c667caaedc090bdd0b5388d5245c72892cac WatchSource:0}: Error finding container 14288b5787536272fe8f860ad007c667caaedc090bdd0b5388d5245c72892cac: Status 404 returned error can't find the container with id 14288b5787536272fe8f860ad007c667caaedc090bdd0b5388d5245c72892cac
Jul 28 19:04:20.996808 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:20.996729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:21.142406 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:21.142374    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc0dd43a-2a93-44f1-915f-0cb98f3ebce5" path="/var/lib/kubelet/pods/fc0dd43a-2a93-44f1-915f-0cb98f3ebce5/volumes"
Jul 28 19:04:21.401354 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:21.401291    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:21.466882 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:21.466844    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:21.572846 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:21.572821    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-29nx8"
Jul 28 19:04:21.709659 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:21.709572    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-vhsn4" event={"ID":"e684e3a5-5494-43a3-884e-655779d70d39","Type":"ContainerStarted","Data":"14288b5787536272fe8f860ad007c667caaedc090bdd0b5388d5245c72892cac"}
Jul 28 19:04:21.805801 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:21.805759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:21.876034 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:21.875994    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:22.002071 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:22.001987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:22.102993 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:22.102965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:23.092524 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:23.092494    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:23.823426 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:23.823397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:25.743025 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:25.742984    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:25.845193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:25.845160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:25.915408 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:25.915365    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:26.109530 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.109493    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dbb6d7f79-rrgzt"]
Jul 28 19:04:26.129070 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.129036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.133374 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.133345    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 19:04:26.133514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.133469    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f2vqz\""
Jul 28 19:04:26.133514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.133345    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 19:04:26.134744 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.134554    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 19:04:26.134744 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.134564    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 19:04:26.134744 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.134628    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 19:04:26.142614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.142583    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dbb6d7f79-rrgzt"]
Jul 28 19:04:26.187683 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.187651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-serving-cert\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.187864 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.187698    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggrxc\" (UniqueName: \"kubernetes.io/projected/20b022d9-974d-4c11-a006-c1ffab7febc2-kube-api-access-ggrxc\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.187864 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.187726    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-oauth-serving-cert\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.187864 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.187848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-oauth-config\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.188032 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.187875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-console-config\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.188032 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.187903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-service-ca\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.289172 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.289133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-console-config\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.289379 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.289180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-service-ca\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.289379 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.289207    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-serving-cert\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.289379 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.289233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggrxc\" (UniqueName: \"kubernetes.io/projected/20b022d9-974d-4c11-a006-c1ffab7febc2-kube-api-access-ggrxc\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.289379 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.289260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-oauth-serving-cert\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.289379 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.289335    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-oauth-config\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.290026 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.289987    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-console-config\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.290318 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.290116    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-service-ca\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.290560 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.290533    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-oauth-serving-cert\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.292226 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.292204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-oauth-config\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.292325 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.292310    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-serving-cert\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.302701 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.302680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggrxc\" (UniqueName: \"kubernetes.io/projected/20b022d9-974d-4c11-a006-c1ffab7febc2-kube-api-access-ggrxc\") pod \"console-7dbb6d7f79-rrgzt\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") " pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.441240 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.441164    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:26.616902 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.616871    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dbb6d7f79-rrgzt"]
Jul 28 19:04:26.621183 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:04:26.621153    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20b022d9_974d_4c11_a006_c1ffab7febc2.slice/crio-4cac7d003188335e0884629b78dce1ab8d549f050b0beaadbb26265a4313798b WatchSource:0}: Error finding container 4cac7d003188335e0884629b78dce1ab8d549f050b0beaadbb26265a4313798b: Status 404 returned error can't find the container with id 4cac7d003188335e0884629b78dce1ab8d549f050b0beaadbb26265a4313798b
Jul 28 19:04:26.732124 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:26.732039    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbb6d7f79-rrgzt" event={"ID":"20b022d9-974d-4c11-a006-c1ffab7febc2","Type":"ContainerStarted","Data":"4cac7d003188335e0884629b78dce1ab8d549f050b0beaadbb26265a4313798b"}
Jul 28 19:04:28.142514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:28.142481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:28.245057 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:28.244987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:30.755266 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:30.755224    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbb6d7f79-rrgzt" event={"ID":"20b022d9-974d-4c11-a006-c1ffab7febc2","Type":"ContainerStarted","Data":"55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b"}
Jul 28 19:04:30.774781 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:30.774733    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dbb6d7f79-rrgzt" podStartSLOduration=1.70960893 podStartE2EDuration="4.774717867s" totalImagesPullingTime="3.065108937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:30.773518357 +0000 UTC m=+112.235126064" watchObservedRunningTime="2026-07-28 19:04:30.774717867 +0000 UTC m=+112.236325567"
Jul 28 19:04:31.992551 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:31.992504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:32.105705 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:32.105661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:32.239224 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:32.239193    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:32.311699 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:32.311622    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:35.614447 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.614414    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-768ff78c98-vwzj6"]
Jul 28 19:04:35.658150 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.658104    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768ff78c98-vwzj6"]
Jul 28 19:04:35.658312 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.658253    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.674047 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.674017    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 19:04:35.763041 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.763000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-trusted-ca-bundle\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.763237 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.763053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-oauth-serving-cert\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.763237 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.763077    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-serving-cert\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.763237 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.763164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-service-ca\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.763237 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.763209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-oauth-config\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.763237 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.763235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-config\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.763528 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.763262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgl59\" (UniqueName: \"kubernetes.io/projected/8c7326e9-2359-408c-b1a3-59a157b33ec7-kube-api-access-xgl59\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.863939 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.863871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-service-ca\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.864135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.863947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-oauth-config\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.864135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.863976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-config\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.864135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.863995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgl59\" (UniqueName: \"kubernetes.io/projected/8c7326e9-2359-408c-b1a3-59a157b33ec7-kube-api-access-xgl59\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.864135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.864087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-trusted-ca-bundle\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.864135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.864134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-oauth-serving-cert\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.864392 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.864172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-serving-cert\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.864761 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.864695    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-service-ca\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.865190 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.865166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-oauth-serving-cert\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.865270 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.865190    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-config\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.865578 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.865548    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-trusted-ca-bundle\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.867219 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.867176    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-serving-cert\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.867520 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.867494    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-oauth-config\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.873499 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.873470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgl59\" (UniqueName: \"kubernetes.io/projected/8c7326e9-2359-408c-b1a3-59a157b33ec7-kube-api-access-xgl59\") pod \"console-768ff78c98-vwzj6\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") " pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:35.969693 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:35.969657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:36.024515 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:36.024481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:36.121910 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:36.121802    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:36.441737 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:36.441657    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:36.441906 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:36.441771    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:36.448586 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:36.448562    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:36.782696 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:36.782615    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:04:38.501231 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:38.501200    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768ff78c98-vwzj6"]
Jul 28 19:04:38.504266 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:04:38.504230    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7326e9_2359_408c_b1a3_59a157b33ec7.slice/crio-9885081b86cf0cc10a5a9fcf1878ba9577b60d17b8b63035ab460cf28c549483 WatchSource:0}: Error finding container 9885081b86cf0cc10a5a9fcf1878ba9577b60d17b8b63035ab460cf28c549483: Status 404 returned error can't find the container with id 9885081b86cf0cc10a5a9fcf1878ba9577b60d17b8b63035ab460cf28c549483
Jul 28 19:04:38.789334 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:38.789241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-vhsn4" event={"ID":"e684e3a5-5494-43a3-884e-655779d70d39","Type":"ContainerStarted","Data":"e414f3944ed6c79691fed94a75285aa8720e7437dc7baccae0dff3c205fec97a"}
Jul 28 19:04:38.789508 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:38.789400    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-vhsn4"
Jul 28 19:04:38.791227 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:38.791156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768ff78c98-vwzj6" event={"ID":"8c7326e9-2359-408c-b1a3-59a157b33ec7","Type":"ContainerStarted","Data":"80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa"}
Jul 28 19:04:38.791227 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:38.791200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768ff78c98-vwzj6" event={"ID":"8c7326e9-2359-408c-b1a3-59a157b33ec7","Type":"ContainerStarted","Data":"9885081b86cf0cc10a5a9fcf1878ba9577b60d17b8b63035ab460cf28c549483"}
Jul 28 19:04:38.807745 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:38.807713    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-vhsn4"
Jul 28 19:04:38.812544 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:38.812491    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-vhsn4" podStartSLOduration=1.138479258 podStartE2EDuration="18.812475147s" totalImagesPullingTime="17.673995889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:38.81049179 +0000 UTC m=+120.272099493" watchObservedRunningTime="2026-07-28 19:04:38.812475147 +0000 UTC m=+120.274082851"
Jul 28 19:04:38.852416 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:38.852351    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-768ff78c98-vwzj6" podStartSLOduration=3.852327086 podStartE2EDuration="3.852327086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:38.8504825 +0000 UTC m=+120.312090205" watchObservedRunningTime="2026-07-28 19:04:38.852327086 +0000 UTC m=+120.313934790"
Jul 28 19:04:40.092866 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:40.092821    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:40.155057 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:40.155021    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:44.032973 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:44.029598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:44.130637 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:44.130591    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:45.690807 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:45.690770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:45.927491 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:45.927458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:45.970665 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:45.970576    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:45.970665 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:45.970644    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:45.976297 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:45.976274    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:46.423308 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:46.423269    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:46.492520 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:46.492484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:46.828732 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:46.828705    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:04:46.879722 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:46.879688    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dbb6d7f79-rrgzt"]
Jul 28 19:04:46.933112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:46.933082    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:46.997694 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:46.997654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:47.439033 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:47.438999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:47.506579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:47.506547    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:48.834305 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:48.834271    2563 generic.go:356] "Generic (PLEG): container finished" podID="8e7a991a-d67b-44a5-9be4-ad06be8c9d1e" containerID="acf04e669ccae04789e6603d18d7db5b2064027602fabdea38ec1e4c2227f97c" exitCode=0
Jul 28 19:04:48.834734 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:48.834348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bnnpr" event={"ID":"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e","Type":"ContainerDied","Data":"acf04e669ccae04789e6603d18d7db5b2064027602fabdea38ec1e4c2227f97c"}
Jul 28 19:04:48.834734 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:48.834708    2563 scope.go:117] "RemoveContainer" containerID="acf04e669ccae04789e6603d18d7db5b2064027602fabdea38ec1e4c2227f97c"
Jul 28 19:04:49.757070 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:49.757041    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mj6fw_daa9a9e1-99e7-4bd1-9745-9231f6f83ba5/serve-healthcheck-canary/0.log"
Jul 28 19:04:49.840143 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:49.840111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bnnpr" event={"ID":"8e7a991a-d67b-44a5-9be4-ad06be8c9d1e","Type":"ContainerStarted","Data":"465c89ce9770be03aa85aef97992449e4446e8144e136385181847cc8c45b3e0"}
Jul 28 19:04:55.425121 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:55.425089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:55.527854 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:55.527824    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:55.727553 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:55.727478    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:55.828521 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:55.828485    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:04:56.428737 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:04:56.428708    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:05:01.935734 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:01.935702    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:05:02.040296 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:02.040258    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:05:11.903627 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:11.903587    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7dbb6d7f79-rrgzt" podUID="20b022d9-974d-4c11-a006-c1ffab7febc2" containerName="console" containerID="cri-o://55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b" gracePeriod=15
Jul 28 19:05:12.180490 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.180428    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:05:12.362234 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.362198    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-console-config\") pod \"20b022d9-974d-4c11-a006-c1ffab7febc2\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") "
Jul 28 19:05:12.362234 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.362235    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-oauth-serving-cert\") pod \"20b022d9-974d-4c11-a006-c1ffab7febc2\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") "
Jul 28 19:05:12.362459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.362272    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-oauth-config\") pod \"20b022d9-974d-4c11-a006-c1ffab7febc2\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") "
Jul 28 19:05:12.362459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.362312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-service-ca\") pod \"20b022d9-974d-4c11-a006-c1ffab7febc2\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") "
Jul 28 19:05:12.362459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.362359    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggrxc\" (UniqueName: \"kubernetes.io/projected/20b022d9-974d-4c11-a006-c1ffab7febc2-kube-api-access-ggrxc\") pod \"20b022d9-974d-4c11-a006-c1ffab7febc2\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") "
Jul 28 19:05:12.362459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.362387    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-serving-cert\") pod \"20b022d9-974d-4c11-a006-c1ffab7febc2\" (UID: \"20b022d9-974d-4c11-a006-c1ffab7febc2\") "
Jul 28 19:05:12.362715 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.362683    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20b022d9-974d-4c11-a006-c1ffab7febc2" (UID: "20b022d9-974d-4c11-a006-c1ffab7febc2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:12.362715 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.362682    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-console-config" (OuterVolumeSpecName: "console-config") pod "20b022d9-974d-4c11-a006-c1ffab7febc2" (UID: "20b022d9-974d-4c11-a006-c1ffab7febc2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:12.362863 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.362743    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-service-ca" (OuterVolumeSpecName: "service-ca") pod "20b022d9-974d-4c11-a006-c1ffab7febc2" (UID: "20b022d9-974d-4c11-a006-c1ffab7febc2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:12.364833 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.364802    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20b022d9-974d-4c11-a006-c1ffab7febc2" (UID: "20b022d9-974d-4c11-a006-c1ffab7febc2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:12.364953 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.364837    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20b022d9-974d-4c11-a006-c1ffab7febc2" (UID: "20b022d9-974d-4c11-a006-c1ffab7febc2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:12.364953 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.364899    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20b022d9-974d-4c11-a006-c1ffab7febc2-kube-api-access-ggrxc" (OuterVolumeSpecName: "kube-api-access-ggrxc") pod "20b022d9-974d-4c11-a006-c1ffab7febc2" (UID: "20b022d9-974d-4c11-a006-c1ffab7febc2"). InnerVolumeSpecName "kube-api-access-ggrxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:05:12.463790 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.463708    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-oauth-config\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:05:12.463790 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.463738    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-service-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:05:12.463790 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.463749    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggrxc\" (UniqueName: \"kubernetes.io/projected/20b022d9-974d-4c11-a006-c1ffab7febc2-kube-api-access-ggrxc\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:05:12.463790 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.463758    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b022d9-974d-4c11-a006-c1ffab7febc2-console-serving-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:05:12.463790 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.463766    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-console-config\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:05:12.463790 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.463774    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20b022d9-974d-4c11-a006-c1ffab7febc2-oauth-serving-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:05:12.924261 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.924223    2563 generic.go:356] "Generic (PLEG): container finished" podID="20b022d9-974d-4c11-a006-c1ffab7febc2" containerID="55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b" exitCode=0
Jul 28 19:05:12.924653 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.924275    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbb6d7f79-rrgzt"
Jul 28 19:05:12.924653 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.924309    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbb6d7f79-rrgzt" event={"ID":"20b022d9-974d-4c11-a006-c1ffab7febc2","Type":"ContainerDied","Data":"55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b"}
Jul 28 19:05:12.924653 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.924363    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbb6d7f79-rrgzt" event={"ID":"20b022d9-974d-4c11-a006-c1ffab7febc2","Type":"ContainerDied","Data":"4cac7d003188335e0884629b78dce1ab8d549f050b0beaadbb26265a4313798b"}
Jul 28 19:05:12.924653 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.924383    2563 scope.go:117] "RemoveContainer" containerID="55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b"
Jul 28 19:05:12.932890 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.932871    2563 scope.go:117] "RemoveContainer" containerID="55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b"
Jul 28 19:05:12.933162 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:05:12.933145    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b\": container with ID starting with 55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b not found: ID does not exist" containerID="55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b"
Jul 28 19:05:12.933228 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.933169    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b"} err="failed to get container status \"55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b\": rpc error: code = NotFound desc = could not find container \"55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b\": container with ID starting with 55755b0bedb653119f29a9e49e42e689094db7060a8d389119c5e0e85442f46b not found: ID does not exist"
Jul 28 19:05:12.946407 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.946379    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dbb6d7f79-rrgzt"]
Jul 28 19:05:12.950738 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:12.950716    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dbb6d7f79-rrgzt"]
Jul 28 19:05:13.142096 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:13.142065    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20b022d9-974d-4c11-a006-c1ffab7febc2" path="/var/lib/kubelet/pods/20b022d9-974d-4c11-a006-c1ffab7febc2/volumes"
Jul 28 19:05:19.678264 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:19.678223    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:05:25.974581 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:25.974545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:05:26.073647 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:26.073608    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:05:31.395684 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:31.395600    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:05:37.893041 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:37.893008    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:05:37.954697 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:37.954656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:05:46.979858 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:46.979818    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:05:47.067987 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:05:47.067940    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:00.490283 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:00.490244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:03.679665 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.677664    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d4cfdbb4f-48z5t"]
Jul 28 19:06:03.679665 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.678302    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20b022d9-974d-4c11-a006-c1ffab7febc2" containerName="console"
Jul 28 19:06:03.679665 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.678323    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="20b022d9-974d-4c11-a006-c1ffab7febc2" containerName="console"
Jul 28 19:06:03.679665 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.678426    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="20b022d9-974d-4c11-a006-c1ffab7febc2" containerName="console"
Jul 28 19:06:03.681083 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.681044    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.700061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.700017    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d4cfdbb4f-48z5t"]
Jul 28 19:06:03.740065 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.740027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tffzr\" (UniqueName: \"kubernetes.io/projected/964bd10d-f6f5-4d01-b30f-267a43daa8f4-kube-api-access-tffzr\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.740252 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.740079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-serving-cert\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.740252 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.740105    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-service-ca\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.740252 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.740202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-oauth-config\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.740252 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.740231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-oauth-serving-cert\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.740388 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.740260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-trusted-ca-bundle\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.740388 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.740325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-config\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.756613 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.756584    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:03.820912 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.820880    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:03.841038 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-oauth-config\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.841121 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-oauth-serving-cert\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.841121 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-trusted-ca-bundle\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.841121 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-config\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.841242 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tffzr\" (UniqueName: \"kubernetes.io/projected/964bd10d-f6f5-4d01-b30f-267a43daa8f4-kube-api-access-tffzr\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.841242 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-serving-cert\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.841242 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-service-ca\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.841820 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841798    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-oauth-serving-cert\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.841963 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-config\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.842017 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841991    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-trusted-ca-bundle\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.842054 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.841991    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-service-ca\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.844453 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.844433    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-oauth-config\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.844522 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.844452    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-serving-cert\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.850887 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.850864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tffzr\" (UniqueName: \"kubernetes.io/projected/964bd10d-f6f5-4d01-b30f-267a43daa8f4-kube-api-access-tffzr\") pod \"console-7d4cfdbb4f-48z5t\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") " pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:03.880984 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.880948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:03.991028 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:03.990907    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:04.129907 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:04.129867    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d4cfdbb4f-48z5t"]
Jul 28 19:06:04.133951 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:06:04.133896    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod964bd10d_f6f5_4d01_b30f_267a43daa8f4.slice/crio-fb7fec247dc5d8b515016e3f4bf0e7bc62bd04de4104e385630856be2c8556fe WatchSource:0}: Error finding container fb7fec247dc5d8b515016e3f4bf0e7bc62bd04de4104e385630856be2c8556fe: Status 404 returned error can't find the container with id fb7fec247dc5d8b515016e3f4bf0e7bc62bd04de4104e385630856be2c8556fe
Jul 28 19:06:04.473331 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:04.473292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:04.539357 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:04.539315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:05.102156 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:05.102122    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4cfdbb4f-48z5t" event={"ID":"964bd10d-f6f5-4d01-b30f-267a43daa8f4","Type":"ContainerStarted","Data":"2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2"}
Jul 28 19:06:05.102529 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:05.102180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4cfdbb4f-48z5t" event={"ID":"964bd10d-f6f5-4d01-b30f-267a43daa8f4","Type":"ContainerStarted","Data":"fb7fec247dc5d8b515016e3f4bf0e7bc62bd04de4104e385630856be2c8556fe"}
Jul 28 19:06:05.122232 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:05.122183    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d4cfdbb4f-48z5t" podStartSLOduration=2.122166325 podStartE2EDuration="2.122166325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:06:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:06:05.121196041 +0000 UTC m=+206.582803743" watchObservedRunningTime="2026-07-28 19:06:05.122166325 +0000 UTC m=+206.583774028"
Jul 28 19:06:13.991395 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:13.991354    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:13.991795 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:13.991432    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:13.996270 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:13.996248    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:14.137499 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:14.137469    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:06:14.184217 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:14.184186    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768ff78c98-vwzj6"]
Jul 28 19:06:14.289356 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:14.289266    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:14.343880 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:14.343846    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:14.402131 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:14.402094    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:14.508074 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:14.508038    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:14.606004 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:14.605971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:19.821765 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:19.821731    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:36.419519 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:36.419482    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:37.266230 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:37.266195    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:39.204444 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.204385    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-768ff78c98-vwzj6" podUID="8c7326e9-2359-408c-b1a3-59a157b33ec7" containerName="console" containerID="cri-o://80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa" gracePeriod=15
Jul 28 19:06:39.439515 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.439493    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:06:39.520279 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.520188    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-oauth-config\") pod \"8c7326e9-2359-408c-b1a3-59a157b33ec7\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") "
Jul 28 19:06:39.520279 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.520230    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-trusted-ca-bundle\") pod \"8c7326e9-2359-408c-b1a3-59a157b33ec7\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") "
Jul 28 19:06:39.520279 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.520252    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgl59\" (UniqueName: \"kubernetes.io/projected/8c7326e9-2359-408c-b1a3-59a157b33ec7-kube-api-access-xgl59\") pod \"8c7326e9-2359-408c-b1a3-59a157b33ec7\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") "
Jul 28 19:06:39.520279 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.520276    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-serving-cert\") pod \"8c7326e9-2359-408c-b1a3-59a157b33ec7\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") "
Jul 28 19:06:39.520611 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.520393    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-config\") pod \"8c7326e9-2359-408c-b1a3-59a157b33ec7\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") "
Jul 28 19:06:39.520611 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.520437    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-oauth-serving-cert\") pod \"8c7326e9-2359-408c-b1a3-59a157b33ec7\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") "
Jul 28 19:06:39.520611 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.520494    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-service-ca\") pod \"8c7326e9-2359-408c-b1a3-59a157b33ec7\" (UID: \"8c7326e9-2359-408c-b1a3-59a157b33ec7\") "
Jul 28 19:06:39.520762 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.520733    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8c7326e9-2359-408c-b1a3-59a157b33ec7" (UID: "8c7326e9-2359-408c-b1a3-59a157b33ec7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:06:39.520882 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.520857    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-config" (OuterVolumeSpecName: "console-config") pod "8c7326e9-2359-408c-b1a3-59a157b33ec7" (UID: "8c7326e9-2359-408c-b1a3-59a157b33ec7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:06:39.520972 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.520947    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8c7326e9-2359-408c-b1a3-59a157b33ec7" (UID: "8c7326e9-2359-408c-b1a3-59a157b33ec7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:06:39.521177 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.521044    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-service-ca" (OuterVolumeSpecName: "service-ca") pod "8c7326e9-2359-408c-b1a3-59a157b33ec7" (UID: "8c7326e9-2359-408c-b1a3-59a157b33ec7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:06:39.522601 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.522571    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8c7326e9-2359-408c-b1a3-59a157b33ec7" (UID: "8c7326e9-2359-408c-b1a3-59a157b33ec7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:06:39.522699 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.522632    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8c7326e9-2359-408c-b1a3-59a157b33ec7" (UID: "8c7326e9-2359-408c-b1a3-59a157b33ec7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:06:39.522699 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.522653    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c7326e9-2359-408c-b1a3-59a157b33ec7-kube-api-access-xgl59" (OuterVolumeSpecName: "kube-api-access-xgl59") pod "8c7326e9-2359-408c-b1a3-59a157b33ec7" (UID: "8c7326e9-2359-408c-b1a3-59a157b33ec7"). InnerVolumeSpecName "kube-api-access-xgl59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:06:39.621878 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.621839    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-service-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:06:39.621878 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.621873    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-oauth-config\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:06:39.621878 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.621883    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-trusted-ca-bundle\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:06:39.621878 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.621892    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgl59\" (UniqueName: \"kubernetes.io/projected/8c7326e9-2359-408c-b1a3-59a157b33ec7-kube-api-access-xgl59\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:06:39.622156 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.621901    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-serving-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:06:39.622156 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.621912    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-console-config\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:06:39.622156 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:39.621945    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c7326e9-2359-408c-b1a3-59a157b33ec7-oauth-serving-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:06:40.225885 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.225844    2563 generic.go:356] "Generic (PLEG): container finished" podID="8c7326e9-2359-408c-b1a3-59a157b33ec7" containerID="80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa" exitCode=0
Jul 28 19:06:40.226305 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.225897    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768ff78c98-vwzj6"
Jul 28 19:06:40.226305 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.225899    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768ff78c98-vwzj6" event={"ID":"8c7326e9-2359-408c-b1a3-59a157b33ec7","Type":"ContainerDied","Data":"80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa"}
Jul 28 19:06:40.226305 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.226023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768ff78c98-vwzj6" event={"ID":"8c7326e9-2359-408c-b1a3-59a157b33ec7","Type":"ContainerDied","Data":"9885081b86cf0cc10a5a9fcf1878ba9577b60d17b8b63035ab460cf28c549483"}
Jul 28 19:06:40.226305 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.226050    2563 scope.go:117] "RemoveContainer" containerID="80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa"
Jul 28 19:06:40.235222 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.235205    2563 scope.go:117] "RemoveContainer" containerID="80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa"
Jul 28 19:06:40.235522 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:06:40.235504    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa\": container with ID starting with 80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa not found: ID does not exist" containerID="80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa"
Jul 28 19:06:40.235583 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.235533    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa"} err="failed to get container status \"80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa\": rpc error: code = NotFound desc = could not find container \"80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa\": container with ID starting with 80aba9178d950e352f7b6a9537bcbb6777ee8ab11cdb6b52b452e110cb4ed3fa not found: ID does not exist"
Jul 28 19:06:40.247996 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.247964    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768ff78c98-vwzj6"]
Jul 28 19:06:40.255214 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.255180    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-768ff78c98-vwzj6"]
Jul 28 19:06:40.462396 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.462368    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:40.549869 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:40.549785    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:41.142705 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:41.142670    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c7326e9-2359-408c-b1a3-59a157b33ec7" path="/var/lib/kubelet/pods/8c7326e9-2359-408c-b1a3-59a157b33ec7/volumes"
Jul 28 19:06:42.111585 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:42.111552    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:42.216781 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:42.216744    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:42.632554 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:42.632517    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:42.744707 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:42.744676    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:44.335084 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:44.335048    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:44.509497 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:44.509455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:47.070058 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:47.070025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:47.172437 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:47.172401    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:48.880226 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:48.880184    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:48.981852 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:48.981799    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:52.514015 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:52.513982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:53.299845 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.299813    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s4q2b"]
Jul 28 19:06:53.300077 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.300066    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c7326e9-2359-408c-b1a3-59a157b33ec7" containerName="console"
Jul 28 19:06:53.300134 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.300079    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c7326e9-2359-408c-b1a3-59a157b33ec7" containerName="console"
Jul 28 19:06:53.300134 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.300131    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c7326e9-2359-408c-b1a3-59a157b33ec7" containerName="console"
Jul 28 19:06:53.302741 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.302725    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-s4q2b"
Jul 28 19:06:53.305514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.305491    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 19:06:53.306663 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.306646    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-l95qx\""
Jul 28 19:06:53.306767 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.306684    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 19:06:53.312478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.312458    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s4q2b"]
Jul 28 19:06:53.320256 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.320235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thg5c\" (UniqueName: \"kubernetes.io/projected/d23e2d23-8931-48d9-b6d8-badf760ab347-kube-api-access-thg5c\") pod \"cert-manager-55b4d954d5-s4q2b\" (UID: \"d23e2d23-8931-48d9-b6d8-badf760ab347\") " pod="cert-manager/cert-manager-55b4d954d5-s4q2b"
Jul 28 19:06:53.320368 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.320282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d23e2d23-8931-48d9-b6d8-badf760ab347-bound-sa-token\") pod \"cert-manager-55b4d954d5-s4q2b\" (UID: \"d23e2d23-8931-48d9-b6d8-badf760ab347\") " pod="cert-manager/cert-manager-55b4d954d5-s4q2b"
Jul 28 19:06:53.420778 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.420742    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thg5c\" (UniqueName: \"kubernetes.io/projected/d23e2d23-8931-48d9-b6d8-badf760ab347-kube-api-access-thg5c\") pod \"cert-manager-55b4d954d5-s4q2b\" (UID: \"d23e2d23-8931-48d9-b6d8-badf760ab347\") " pod="cert-manager/cert-manager-55b4d954d5-s4q2b"
Jul 28 19:06:53.420984 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.420789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d23e2d23-8931-48d9-b6d8-badf760ab347-bound-sa-token\") pod \"cert-manager-55b4d954d5-s4q2b\" (UID: \"d23e2d23-8931-48d9-b6d8-badf760ab347\") " pod="cert-manager/cert-manager-55b4d954d5-s4q2b"
Jul 28 19:06:53.429046 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.429024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thg5c\" (UniqueName: \"kubernetes.io/projected/d23e2d23-8931-48d9-b6d8-badf760ab347-kube-api-access-thg5c\") pod \"cert-manager-55b4d954d5-s4q2b\" (UID: \"d23e2d23-8931-48d9-b6d8-badf760ab347\") " pod="cert-manager/cert-manager-55b4d954d5-s4q2b"
Jul 28 19:06:53.429140 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.429021    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d23e2d23-8931-48d9-b6d8-badf760ab347-bound-sa-token\") pod \"cert-manager-55b4d954d5-s4q2b\" (UID: \"d23e2d23-8931-48d9-b6d8-badf760ab347\") " pod="cert-manager/cert-manager-55b4d954d5-s4q2b"
Jul 28 19:06:53.613043 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.613014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-s4q2b"
Jul 28 19:06:53.744385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:53.744246    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s4q2b"]
Jul 28 19:06:53.746858 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:06:53.746825    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd23e2d23_8931_48d9_b6d8_badf760ab347.slice/crio-36d313ae33bc895b2157507987db56b6104e7611796d25dbd290c94744de8eb2 WatchSource:0}: Error finding container 36d313ae33bc895b2157507987db56b6104e7611796d25dbd290c94744de8eb2: Status 404 returned error can't find the container with id 36d313ae33bc895b2157507987db56b6104e7611796d25dbd290c94744de8eb2
Jul 28 19:06:54.282331 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:54.282294    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-s4q2b" event={"ID":"d23e2d23-8931-48d9-b6d8-badf760ab347","Type":"ContainerStarted","Data":"36d313ae33bc895b2157507987db56b6104e7611796d25dbd290c94744de8eb2"}
Jul 28 19:06:54.317585 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:54.317547    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:54.416133 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:54.416102    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:06:57.297519 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:57.297482    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-s4q2b" event={"ID":"d23e2d23-8931-48d9-b6d8-badf760ab347","Type":"ContainerStarted","Data":"86887155b1e32f6b2aec494ec79b7361bd0e30514067c5aba8720c34c0cf62c6"}
Jul 28 19:06:57.329783 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:06:57.329730    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-s4q2b" podStartSLOduration=0.851350813 podStartE2EDuration="4.329714455s" totalImagesPullingTime="3.478363642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:06:57.328102955 +0000 UTC m=+258.789710658" watchObservedRunningTime="2026-07-28 19:06:57.329714455 +0000 UTC m=+258.791322158"
Jul 28 19:07:00.143617 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:00.143537    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:00.242982 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:00.242953    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:01.966528 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:01.966490    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:05.195713 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:05.195680    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:05.315322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:05.315293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:14.815958 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:14.815900    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:14.913441 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:14.913410    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:15.482199 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.482165    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:15.690209 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.690176    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:15.792488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.792414    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:15.832208 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.832178    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"]
Jul 28 19:07:15.834323 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.834309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.836995 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.836972    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 19:07:15.837230 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.837212    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 19:07:15.837328 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.837218    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 19:07:15.837328 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.837281    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2sd7z\""
Jul 28 19:07:15.837328 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.837293    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 19:07:15.853193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.853160    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"]
Jul 28 19:07:15.894449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.894412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fb6e434-4979-4721-b1bc-f75d05197218-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.894623 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.894463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fb6e434-4979-4721-b1bc-f75d05197218-tmp\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.894623 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.894490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-696cn\" (UniqueName: \"kubernetes.io/projected/0fb6e434-4979-4721-b1bc-f75d05197218-kube-api-access-696cn\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.894623 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.894574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0fb6e434-4979-4721-b1bc-f75d05197218-manifests\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.894623 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.894607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0fb6e434-4979-4721-b1bc-f75d05197218-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.966937 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.966881    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:15.995015 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.994973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0fb6e434-4979-4721-b1bc-f75d05197218-manifests\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.995111 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.995030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0fb6e434-4979-4721-b1bc-f75d05197218-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.995111 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.995088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fb6e434-4979-4721-b1bc-f75d05197218-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.995175 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.995141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fb6e434-4979-4721-b1bc-f75d05197218-tmp\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.995175 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.995166    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-696cn\" (UniqueName: \"kubernetes.io/projected/0fb6e434-4979-4721-b1bc-f75d05197218-kube-api-access-696cn\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.995467 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.995447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0fb6e434-4979-4721-b1bc-f75d05197218-manifests\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.997617 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.997595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fb6e434-4979-4721-b1bc-f75d05197218-tmp\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.997746 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.997683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fb6e434-4979-4721-b1bc-f75d05197218-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:15.997746 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:15.997713    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0fb6e434-4979-4721-b1bc-f75d05197218-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:16.009319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:16.009296    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-696cn\" (UniqueName: \"kubernetes.io/projected/0fb6e434-4979-4721-b1bc-f75d05197218-kube-api-access-696cn\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb\" (UID: \"0fb6e434-4979-4721-b1bc-f75d05197218\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:16.070188 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:16.070156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:16.144648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:16.144613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:16.169211 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:16.169185    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:16.276363 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:16.276332    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:16.293648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:16.293618    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"]
Jul 28 19:07:16.298990 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:07:16.298961    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb6e434_4979_4721_b1bc_f75d05197218.slice/crio-b69922a019cefac3d03f2f95f4cf1f0135b8a79967790f23e574e966eddd26bc WatchSource:0}: Error finding container b69922a019cefac3d03f2f95f4cf1f0135b8a79967790f23e574e966eddd26bc: Status 404 returned error can't find the container with id b69922a019cefac3d03f2f95f4cf1f0135b8a79967790f23e574e966eddd26bc
Jul 28 19:07:16.368705 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:16.368618    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb" event={"ID":"0fb6e434-4979-4721-b1bc-f75d05197218","Type":"ContainerStarted","Data":"b69922a019cefac3d03f2f95f4cf1f0135b8a79967790f23e574e966eddd26bc"}
Jul 28 19:07:16.371270 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:16.371242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:20.388035 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:20.388000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb" event={"ID":"0fb6e434-4979-4721-b1bc-f75d05197218","Type":"ContainerStarted","Data":"7f373182cbab8e60ce94f4ef234dab083ccc1893c8688e28077eafc85ac5b892"}
Jul 28 19:07:21.394082 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:21.394051    2563 generic.go:356] "Generic (PLEG): container finished" podID="0fb6e434-4979-4721-b1bc-f75d05197218" containerID="7f373182cbab8e60ce94f4ef234dab083ccc1893c8688e28077eafc85ac5b892" exitCode=0
Jul 28 19:07:21.394460 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:21.394096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb" event={"ID":"0fb6e434-4979-4721-b1bc-f75d05197218","Type":"ContainerDied","Data":"7f373182cbab8e60ce94f4ef234dab083ccc1893c8688e28077eafc85ac5b892"}
Jul 28 19:07:22.399505 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:22.399469    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb" event={"ID":"0fb6e434-4979-4721-b1bc-f75d05197218","Type":"ContainerStarted","Data":"9173dd86fdbc0db43ddc6dbf72b486f322d7ea3d3a051a2490eef1f26349c50a"}
Jul 28 19:07:22.399908 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:22.399597    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:22.423411 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:22.423365    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb" podStartSLOduration=2.980809812 podStartE2EDuration="7.423349698s" totalImagesPullingTime="3.365403998s" totalInitContainerRuntime="1.077135888s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:22.421999353 +0000 UTC m=+283.883607060" watchObservedRunningTime="2026-07-28 19:07:22.423349698 +0000 UTC m=+283.884957401"
Jul 28 19:07:31.575028 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:31.574983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:31.677315 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:31.677277    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:33.406169 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.406137    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb"
Jul 28 19:07:33.575716 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.575683    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"]
Jul 28 19:07:33.579260 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.579245    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.582151 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.582132    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 19:07:33.582272 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.582254    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 19:07:33.582422 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.582408    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:07:33.582599 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.582582    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 19:07:33.582816 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.582797    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 19:07:33.582864 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.582797    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kk6wv\""
Jul 28 19:07:33.588677 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.588657    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"]
Jul 28 19:07:33.741418 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.741329    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cf0dae32-852c-417c-8385-46de0f25cd2c-manager-config\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.741418 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.741369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf0dae32-852c-417c-8385-46de0f25cd2c-metrics-cert\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.741418 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.741393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf0dae32-852c-417c-8385-46de0f25cd2c-cert\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.741643 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.741462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rglhz\" (UniqueName: \"kubernetes.io/projected/cf0dae32-852c-417c-8385-46de0f25cd2c-kube-api-access-rglhz\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.842822 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.842772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cf0dae32-852c-417c-8385-46de0f25cd2c-manager-config\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.842822 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.842829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf0dae32-852c-417c-8385-46de0f25cd2c-metrics-cert\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.843087 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.842864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf0dae32-852c-417c-8385-46de0f25cd2c-cert\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.843087 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.842902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rglhz\" (UniqueName: \"kubernetes.io/projected/cf0dae32-852c-417c-8385-46de0f25cd2c-kube-api-access-rglhz\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.843449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.843429    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cf0dae32-852c-417c-8385-46de0f25cd2c-manager-config\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.845631 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.845609    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf0dae32-852c-417c-8385-46de0f25cd2c-cert\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.845631 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.845629    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf0dae32-852c-417c-8385-46de0f25cd2c-metrics-cert\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.851480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.851453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rglhz\" (UniqueName: \"kubernetes.io/projected/cf0dae32-852c-417c-8385-46de0f25cd2c-kube-api-access-rglhz\") pod \"lws-controller-manager-7fbf845658-kdgzf\" (UID: \"cf0dae32-852c-417c-8385-46de0f25cd2c\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:33.888582 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:33.888530    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:34.025305 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:34.025282    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"]
Jul 28 19:07:34.027581 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:07:34.027555    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf0dae32_852c_417c_8385_46de0f25cd2c.slice/crio-09811aa22afccdfef793c4af73af60bd4e6389f7e5c522dcfe1e7c04b3f4aec9 WatchSource:0}: Error finding container 09811aa22afccdfef793c4af73af60bd4e6389f7e5c522dcfe1e7c04b3f4aec9: Status 404 returned error can't find the container with id 09811aa22afccdfef793c4af73af60bd4e6389f7e5c522dcfe1e7c04b3f4aec9
Jul 28 19:07:34.447379 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:34.447348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf" event={"ID":"cf0dae32-852c-417c-8385-46de0f25cd2c","Type":"ContainerStarted","Data":"09811aa22afccdfef793c4af73af60bd4e6389f7e5c522dcfe1e7c04b3f4aec9"}
Jul 28 19:07:35.096171 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:35.096073    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:35.204439 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:35.204396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:35.679583 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:35.679545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:36.192323 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:36.192288    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:36.386047 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:36.386011    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:36.483111 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:36.483081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:36.713233 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:36.713138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:37.147294 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.147267    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:37.253345 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.253312    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:37.357493 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.357455    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gmf4z"]
Jul 28 19:07:37.359498 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.359473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:37.359654 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.359550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z"
Jul 28 19:07:37.362807 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.362787    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 19:07:37.374261 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.374235    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gmf4z"]
Jul 28 19:07:37.442569 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.442471    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:37.461787 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.461755    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf" event={"ID":"cf0dae32-852c-417c-8385-46de0f25cd2c","Type":"ContainerStarted","Data":"83a41f59c8135ea5dbe402f49769441285d435b0d519c4e59e0e454bf8e9ba51"}
Jul 28 19:07:37.461960 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.461890    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:37.476094 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.476065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a7bbf4-6132-442d-ac6b-126910605785-tls-certs\") pod \"model-serving-api-54d89bd5fc-gmf4z\" (UID: \"81a7bbf4-6132-442d-ac6b-126910605785\") " pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z"
Jul 28 19:07:37.476169 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.476097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpgpb\" (UniqueName: \"kubernetes.io/projected/81a7bbf4-6132-442d-ac6b-126910605785-kube-api-access-zpgpb\") pod \"model-serving-api-54d89bd5fc-gmf4z\" (UID: \"81a7bbf4-6132-442d-ac6b-126910605785\") " pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z"
Jul 28 19:07:37.480497 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.480449    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf" podStartSLOduration=2.106361628 podStartE2EDuration="4.480433594s" totalImagesPullingTime="2.374071966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:37.479580146 +0000 UTC m=+298.941187850" watchObservedRunningTime="2026-07-28 19:07:37.480433594 +0000 UTC m=+298.942041296"
Jul 28 19:07:37.577198 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.577158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a7bbf4-6132-442d-ac6b-126910605785-tls-certs\") pod \"model-serving-api-54d89bd5fc-gmf4z\" (UID: \"81a7bbf4-6132-442d-ac6b-126910605785\") " pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z"
Jul 28 19:07:37.577198 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.577200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpgpb\" (UniqueName: \"kubernetes.io/projected/81a7bbf4-6132-442d-ac6b-126910605785-kube-api-access-zpgpb\") pod \"model-serving-api-54d89bd5fc-gmf4z\" (UID: \"81a7bbf4-6132-442d-ac6b-126910605785\") " pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z"
Jul 28 19:07:37.579775 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.579750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81a7bbf4-6132-442d-ac6b-126910605785-tls-certs\") pod \"model-serving-api-54d89bd5fc-gmf4z\" (UID: \"81a7bbf4-6132-442d-ac6b-126910605785\") " pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z"
Jul 28 19:07:37.586562 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.586536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpgpb\" (UniqueName: \"kubernetes.io/projected/81a7bbf4-6132-442d-ac6b-126910605785-kube-api-access-zpgpb\") pod \"model-serving-api-54d89bd5fc-gmf4z\" (UID: \"81a7bbf4-6132-442d-ac6b-126910605785\") " pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z"
Jul 28 19:07:37.669149 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.669113    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z"
Jul 28 19:07:37.843007 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:07:37.842974    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a7bbf4_6132_442d_ac6b_126910605785.slice/crio-45688baa6f1c07053fabde51a9efa4f9d7243681c1280c8d4604741c4f5030f2 WatchSource:0}: Error finding container 45688baa6f1c07053fabde51a9efa4f9d7243681c1280c8d4604741c4f5030f2: Status 404 returned error can't find the container with id 45688baa6f1c07053fabde51a9efa4f9d7243681c1280c8d4604741c4f5030f2
Jul 28 19:07:37.855159 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:37.855132    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gmf4z"]
Jul 28 19:07:38.427219 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:38.427183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:38.466769 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:38.466728    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z" event={"ID":"81a7bbf4-6132-442d-ac6b-126910605785","Type":"ContainerStarted","Data":"45688baa6f1c07053fabde51a9efa4f9d7243681c1280c8d4604741c4f5030f2"}
Jul 28 19:07:38.936410 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:38.936381    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:07:38.939865 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:38.939840    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:07:38.940267 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:38.940252    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 19:07:40.476589 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:40.476549    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z" event={"ID":"81a7bbf4-6132-442d-ac6b-126910605785","Type":"ContainerStarted","Data":"7d16410048ca67a01b444fbdd77d97cba929f6da91df00f1d543f3ef5e79fd90"}
Jul 28 19:07:40.477019 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:40.476632    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z"
Jul 28 19:07:40.503240 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:40.503182    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z" podStartSLOduration=1.5306424 podStartE2EDuration="3.503164693s" totalImagesPullingTime="1.972522293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:40.501321165 +0000 UTC m=+301.962928867" watchObservedRunningTime="2026-07-28 19:07:40.503164693 +0000 UTC m=+301.964772396"
Jul 28 19:07:42.689832 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:42.689794    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:42.786358 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:42.786329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:42.917610 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:42.917577    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"]
Jul 28 19:07:42.919789 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:42.919769    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:42.924122 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:42.924098    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 19:07:42.924268 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:42.924135    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 19:07:42.924268 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:42.924152    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rtl7m\""
Jul 28 19:07:42.924268 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:42.924212    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 19:07:42.934816 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:42.934784    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"]
Jul 28 19:07:42.958709 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:42.958621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:43.020879 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.020834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.021074 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.020985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c10ce11-b2ef-4213-9542-579ff32c8211-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.021074 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.021019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.021074 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.021065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.021238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.021080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.021238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.021099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.021238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.021128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thhms\" (UniqueName: \"kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-kube-api-access-thhms\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.021238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.021180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.021238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.021217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.121806 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.121770    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.121806 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.121805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.121835    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.121968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thhms\" (UniqueName: \"kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-kube-api-access-thhms\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.122014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.122039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.122093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.122146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c10ce11-b2ef-4213-9542-579ff32c8211-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.122189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122249 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.122236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.122442    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122563 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.122545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.122825 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.122803    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.123154 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.123133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c10ce11-b2ef-4213-9542-579ff32c8211-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.124339 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.124319    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.124504 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.124488    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.146691 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.146661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thhms\" (UniqueName: \"kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-kube-api-access-thhms\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.149643 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.149618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.192308 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.192276    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:43.230581 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.230493    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:43.290599 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.290572    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:43.369471 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.369439    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"]
Jul 28 19:07:43.372281 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:07:43.372253    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c10ce11_b2ef_4213_9542_579ff32c8211.slice/crio-f9337d53b35a97fe8543f55541bb42de89027d11eb93561cb65fe49787d18a3a WatchSource:0}: Error finding container f9337d53b35a97fe8543f55541bb42de89027d11eb93561cb65fe49787d18a3a: Status 404 returned error can't find the container with id f9337d53b35a97fe8543f55541bb42de89027d11eb93561cb65fe49787d18a3a
Jul 28 19:07:43.374324 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.374307    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:07:43.488892 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:43.488798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz" event={"ID":"3c10ce11-b2ef-4213-9542-579ff32c8211","Type":"ContainerStarted","Data":"f9337d53b35a97fe8543f55541bb42de89027d11eb93561cb65fe49787d18a3a"}
Jul 28 19:07:44.113770 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.113737    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"]
Jul 28 19:07:44.116297 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.116281    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.119796 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.119765    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2wzxf\""
Jul 28 19:07:44.119953 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.119809    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 19:07:44.119953 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.119822    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 19:07:44.119953 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.119809    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 19:07:44.132029 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.131964    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"]
Jul 28 19:07:44.233029 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.232992    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfjpg\" (UniqueName: \"kubernetes.io/projected/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-kube-api-access-dfjpg\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.233221 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.233073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.233221 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.233110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.233221 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.233138    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.233221 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.233170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.233537 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.233231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.233537 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.233270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.334374 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.334336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.334374 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.334394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfjpg\" (UniqueName: \"kubernetes.io/projected/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-kube-api-access-dfjpg\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.334628 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.334475    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.334628 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.334507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.334628 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.334536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.334628 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.334561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.334830 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.334655    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.336117 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.336078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.337668 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.337616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.338682 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.338625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.339077 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.339036    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.339383 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.339365    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.345424 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.345367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfjpg\" (UniqueName: \"kubernetes.io/projected/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-kube-api-access-dfjpg\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.345732 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.345687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d7aaeb7-658a-4bec-8dda-3164f1f87b92-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7qxw8\" (UID: \"0d7aaeb7-658a-4bec-8dda-3164f1f87b92\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.425514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.425428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:44.732568 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:44.732540    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"]
Jul 28 19:07:44.733932 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:07:44.733886    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d7aaeb7_658a_4bec_8dda_3164f1f87b92.slice/crio-6287a4323277f0ac4d42765ff3365958c645e83e0f1d1aad7377129e7d75610c WatchSource:0}: Error finding container 6287a4323277f0ac4d42765ff3365958c645e83e0f1d1aad7377129e7d75610c: Status 404 returned error can't find the container with id 6287a4323277f0ac4d42765ff3365958c645e83e0f1d1aad7377129e7d75610c
Jul 28 19:07:45.500399 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:45.500354    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8" event={"ID":"0d7aaeb7-658a-4bec-8dda-3164f1f87b92","Type":"ContainerStarted","Data":"6287a4323277f0ac4d42765ff3365958c645e83e0f1d1aad7377129e7d75610c"}
Jul 28 19:07:46.205435 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:46.205398    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:07:46.205504 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:46.205481    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:07:46.205538 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:46.205512    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:07:46.507525 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:46.507406    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz" event={"ID":"3c10ce11-b2ef-4213-9542-579ff32c8211","Type":"ContainerStarted","Data":"9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d"}
Jul 28 19:07:46.541896 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:46.541822    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz" podStartSLOduration=1.71108404 podStartE2EDuration="4.54179825s" totalImagesPullingTime="2.83071421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:46.539622344 +0000 UTC m=+308.001230047" watchObservedRunningTime="2026-07-28 19:07:46.54179825 +0000 UTC m=+308.003405954"
Jul 28 19:07:46.764133 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:46.764042    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:46.965570 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:46.965502    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:47.078366 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:47.078324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:47.231498 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:47.231452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:47.233105 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:47.233078    2563 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Jul 28 19:07:47.233232 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:47.233130    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz" podUID="3c10ce11-b2ef-4213-9542-579ff32c8211" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Jul 28 19:07:47.286936 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:47.286881    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:47.699857 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:47.699822    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"]
Jul 28 19:07:47.921092 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:47.921063    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:07:47.921170 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:47.921128    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:07:48.163713 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:48.163662    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:48.231651 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:48.231561    2563 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Jul 28 19:07:48.231651 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:48.231617    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz" podUID="3c10ce11-b2ef-4213-9542-579ff32c8211" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Jul 28 19:07:48.331839 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:48.331807    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:48.469861 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:48.469829    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-kdgzf"
Jul 28 19:07:48.519480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:48.519382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8" event={"ID":"0d7aaeb7-658a-4bec-8dda-3164f1f87b92","Type":"ContainerStarted","Data":"d0fd90c8c0d6afd4c94eca143037b8b34254137bf55788c48a11a67518469f59"}
Jul 28 19:07:48.559853 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:48.559762    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8" podStartSLOduration=1.375122547 podStartE2EDuration="4.559742927s" totalImagesPullingTime="3.18462038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:48.558509264 +0000 UTC m=+310.020116968" watchObservedRunningTime="2026-07-28 19:07:48.559742927 +0000 UTC m=+310.021350627"
Jul 28 19:07:49.231631 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:49.231595    2563 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Jul 28 19:07:49.232034 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:49.231649    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz" podUID="3c10ce11-b2ef-4213-9542-579ff32c8211" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Jul 28 19:07:49.528392 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:49.528278    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz" podUID="3c10ce11-b2ef-4213-9542-579ff32c8211" containerName="istio-proxy" containerID="cri-o://9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d" gracePeriod=30
Jul 28 19:07:49.528566 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:49.528535    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:49.530044 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:49.530025    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7qxw8"
Jul 28 19:07:49.642570 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:49.642518    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:07:51.484762 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:51.484729    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gmf4z"
Jul 28 19:07:54.776069 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.776045    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:54.820862 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.820841    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-token\") pod \"3c10ce11-b2ef-4213-9542-579ff32c8211\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") "
Jul 28 19:07:54.821048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.820881    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-socket\") pod \"3c10ce11-b2ef-4213-9542-579ff32c8211\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") "
Jul 28 19:07:54.821048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.820906    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-podinfo\") pod \"3c10ce11-b2ef-4213-9542-579ff32c8211\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") "
Jul 28 19:07:54.821048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.820947    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thhms\" (UniqueName: \"kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-kube-api-access-thhms\") pod \"3c10ce11-b2ef-4213-9542-579ff32c8211\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") "
Jul 28 19:07:54.821048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.820975    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-data\") pod \"3c10ce11-b2ef-4213-9542-579ff32c8211\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") "
Jul 28 19:07:54.821048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.820994    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-credential-socket\") pod \"3c10ce11-b2ef-4213-9542-579ff32c8211\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") "
Jul 28 19:07:54.821048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.821009    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-certs\") pod \"3c10ce11-b2ef-4213-9542-579ff32c8211\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") "
Jul 28 19:07:54.821048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.821047    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-envoy\") pod \"3c10ce11-b2ef-4213-9542-579ff32c8211\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") "
Jul 28 19:07:54.821385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.821098    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c10ce11-b2ef-4213-9542-579ff32c8211-istiod-ca-cert\") pod \"3c10ce11-b2ef-4213-9542-579ff32c8211\" (UID: \"3c10ce11-b2ef-4213-9542-579ff32c8211\") "
Jul 28 19:07:54.821385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.821242    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c10ce11-b2ef-4213-9542-579ff32c8211" (UID: "3c10ce11-b2ef-4213-9542-579ff32c8211"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:07:54.821385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.821365    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c10ce11-b2ef-4213-9542-579ff32c8211" (UID: "3c10ce11-b2ef-4213-9542-579ff32c8211"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:07:54.821385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.821366    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c10ce11-b2ef-4213-9542-579ff32c8211" (UID: "3c10ce11-b2ef-4213-9542-579ff32c8211"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:07:54.821588 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.821531    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c10ce11-b2ef-4213-9542-579ff32c8211-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c10ce11-b2ef-4213-9542-579ff32c8211" (UID: "3c10ce11-b2ef-4213-9542-579ff32c8211"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:07:54.821814 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.821788    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c10ce11-b2ef-4213-9542-579ff32c8211" (UID: "3c10ce11-b2ef-4213-9542-579ff32c8211"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:07:54.823307 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.823283    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c10ce11-b2ef-4213-9542-579ff32c8211" (UID: "3c10ce11-b2ef-4213-9542-579ff32c8211"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:07:54.823424 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.823378    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c10ce11-b2ef-4213-9542-579ff32c8211" (UID: "3c10ce11-b2ef-4213-9542-579ff32c8211"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:07:54.823424 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.823400    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c10ce11-b2ef-4213-9542-579ff32c8211" (UID: "3c10ce11-b2ef-4213-9542-579ff32c8211"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:07:54.823531 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.823513    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-kube-api-access-thhms" (OuterVolumeSpecName: "kube-api-access-thhms") pod "3c10ce11-b2ef-4213-9542-579ff32c8211" (UID: "3c10ce11-b2ef-4213-9542-579ff32c8211"). InnerVolumeSpecName "kube-api-access-thhms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:07:54.922564 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.922518    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:07:54.922564 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.922556    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:07:54.922564 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.922568    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:07:54.922811 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.922583    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thhms\" (UniqueName: \"kubernetes.io/projected/3c10ce11-b2ef-4213-9542-579ff32c8211-kube-api-access-thhms\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:07:54.922811 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.922596    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:07:54.922811 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.922607    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:07:54.922811 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.922617    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:07:54.922811 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.922627    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c10ce11-b2ef-4213-9542-579ff32c8211-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:07:54.922811 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:54.922637    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c10ce11-b2ef-4213-9542-579ff32c8211-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:07:55.553650 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:55.553557    2563 generic.go:356] "Generic (PLEG): container finished" podID="3c10ce11-b2ef-4213-9542-579ff32c8211" containerID="9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d" exitCode=0
Jul 28 19:07:55.553650 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:55.553620    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"
Jul 28 19:07:55.553875 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:55.553622    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz" event={"ID":"3c10ce11-b2ef-4213-9542-579ff32c8211","Type":"ContainerDied","Data":"9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d"}
Jul 28 19:07:55.553875 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:55.553748    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz" event={"ID":"3c10ce11-b2ef-4213-9542-579ff32c8211","Type":"ContainerDied","Data":"f9337d53b35a97fe8543f55541bb42de89027d11eb93561cb65fe49787d18a3a"}
Jul 28 19:07:55.553875 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:55.553779    2563 scope.go:117] "RemoveContainer" containerID="9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d"
Jul 28 19:07:55.562458 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:55.562436    2563 scope.go:117] "RemoveContainer" containerID="9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d"
Jul 28 19:07:55.562764 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:07:55.562743    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d\": container with ID starting with 9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d not found: ID does not exist" containerID="9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d"
Jul 28 19:07:55.562855 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:55.562773    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d"} err="failed to get container status \"9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d\": rpc error: code = NotFound desc = could not find container \"9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d\": container with ID starting with 9bd32692ee139b2fb100a1aec1095073510135dbb30e7a1fe6ccb5cfe0d0127d not found: ID does not exist"
Jul 28 19:07:55.574787 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:55.574741    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"]
Jul 28 19:07:55.579274 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:55.579244    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcrc7xz"]
Jul 28 19:07:57.143236 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:07:57.143203    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c10ce11-b2ef-4213-9542-579ff32c8211" path="/var/lib/kubelet/pods/3c10ce11-b2ef-4213-9542-579ff32c8211/volumes"
Jul 28 19:08:36.540790 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:08:36.540671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:08:36.634446 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:08:36.634414    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:08:36.713199 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:08:36.713169    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:08:40.923531 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:08:40.923494    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:08:40.990735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:08:40.990699    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:05.576817 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:05.576782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:09.009126 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:09.009091    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:09.778892 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:09.778851    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:13.068457 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:13.068421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:13.422317 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:13.422231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:15.013027 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:15.012993    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:16.619860 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:16.619827    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:17.864120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:17.864089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:20.561686 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:20.561648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:22.382676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:22.382648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:22.483565 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:22.483533    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:26.372026 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.371994    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:26.469579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.469544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:26.658597 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.658503    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ddbd6c58c-npmmr"]
Jul 28 19:09:26.658899 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.658881    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c10ce11-b2ef-4213-9542-579ff32c8211" containerName="istio-proxy"
Jul 28 19:09:26.659017 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.658903    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c10ce11-b2ef-4213-9542-579ff32c8211" containerName="istio-proxy"
Jul 28 19:09:26.659074 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.659024    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c10ce11-b2ef-4213-9542-579ff32c8211" containerName="istio-proxy"
Jul 28 19:09:26.662120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.662097    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.684599 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.684561    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ddbd6c58c-npmmr"]
Jul 28 19:09:26.749406 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.749377    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:26.765014 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.764988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86916ca0-cd77-47fa-b6f1-703e61defc88-console-serving-cert\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.765146 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.765021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-console-config\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.765146 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.765071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rkds\" (UniqueName: \"kubernetes.io/projected/86916ca0-cd77-47fa-b6f1-703e61defc88-kube-api-access-6rkds\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.765146 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.765094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-service-ca\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.765146 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.765113    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-oauth-serving-cert\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.765146 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.765129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86916ca0-cd77-47fa-b6f1-703e61defc88-console-oauth-config\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.765331 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.765181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-trusted-ca-bundle\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.859246 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.859205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:26.866437 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.866414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rkds\" (UniqueName: \"kubernetes.io/projected/86916ca0-cd77-47fa-b6f1-703e61defc88-kube-api-access-6rkds\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.866528 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.866452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-service-ca\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.866528 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.866480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-oauth-serving-cert\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.866528 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.866497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86916ca0-cd77-47fa-b6f1-703e61defc88-console-oauth-config\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.866528 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.866517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-trusted-ca-bundle\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.866709 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.866554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86916ca0-cd77-47fa-b6f1-703e61defc88-console-serving-cert\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.866709 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.866582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-console-config\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.867326 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.867291    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-service-ca\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.867474 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.867446    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-console-config\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.867610 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.867526    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-oauth-serving-cert\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.867610 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.867566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86916ca0-cd77-47fa-b6f1-703e61defc88-trusted-ca-bundle\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.869172 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.869148    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86916ca0-cd77-47fa-b6f1-703e61defc88-console-oauth-config\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.869332 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.869314    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86916ca0-cd77-47fa-b6f1-703e61defc88-console-serving-cert\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.876377 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.876357    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rkds\" (UniqueName: \"kubernetes.io/projected/86916ca0-cd77-47fa-b6f1-703e61defc88-kube-api-access-6rkds\") pod \"console-7ddbd6c58c-npmmr\" (UID: \"86916ca0-cd77-47fa-b6f1-703e61defc88\") " pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.970598 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.970507    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:26.991289 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:26.991256    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:27.132590 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:27.132564    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ddbd6c58c-npmmr"]
Jul 28 19:09:27.135512 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:09:27.135486    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86916ca0_cd77_47fa_b6f1_703e61defc88.slice/crio-c56e18aa4f41ff154d065dc3c5dabb4412cbd28648faeb2361a6f43e030ed637 WatchSource:0}: Error finding container c56e18aa4f41ff154d065dc3c5dabb4412cbd28648faeb2361a6f43e030ed637: Status 404 returned error can't find the container with id c56e18aa4f41ff154d065dc3c5dabb4412cbd28648faeb2361a6f43e030ed637
Jul 28 19:09:27.478268 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:27.478241    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:27.590614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:27.590586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:27.936314 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:27.936277    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddbd6c58c-npmmr" event={"ID":"86916ca0-cd77-47fa-b6f1-703e61defc88","Type":"ContainerStarted","Data":"3f80f2750f640fedf9f182def4f4e0484a545c2edaef59a14f5d17acaad94c7f"}
Jul 28 19:09:27.936504 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:27.936331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddbd6c58c-npmmr" event={"ID":"86916ca0-cd77-47fa-b6f1-703e61defc88","Type":"ContainerStarted","Data":"c56e18aa4f41ff154d065dc3c5dabb4412cbd28648faeb2361a6f43e030ed637"}
Jul 28 19:09:27.980939 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:27.980860    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7ddbd6c58c-npmmr" podStartSLOduration=1.980845038 podStartE2EDuration="1.980845038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:09:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:09:27.980723772 +0000 UTC m=+409.442331663" watchObservedRunningTime="2026-07-28 19:09:27.980845038 +0000 UTC m=+409.442452740"
Jul 28 19:09:29.729494 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:29.729458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:29.924947 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:29.924895    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:30.377164 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:30.377128    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:30.487223 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:30.487188    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:34.948005 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:34.947973    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:35.161478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:35.161442    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:36.971303 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:36.971276    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:36.971715 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:36.971335    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:36.976052 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:36.976032    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:37.979614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:37.979585    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ddbd6c58c-npmmr"
Jul 28 19:09:38.033253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:38.033223    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d4cfdbb4f-48z5t"]
Jul 28 19:09:38.168527 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:38.168489    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:38.269415 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:38.269285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:38.372787 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:38.372759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:38.474323 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:38.474293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:41.930647 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:41.930608    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:42.062342 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:42.062310    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:45.647803 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:45.647772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:45.707131 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:45.707096    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:46.242225 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:46.242192    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:46.305760 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:46.305731    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:50.865077 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:50.865037    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:50.966295 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:50.966262    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:55.259232 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:55.259192    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:09:55.359901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:09:55.359865    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:03.053934 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.053827    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d4cfdbb4f-48z5t" podUID="964bd10d-f6f5-4d01-b30f-267a43daa8f4" containerName="console" containerID="cri-o://2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2" gracePeriod=15
Jul 28 19:10:03.301024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.301000    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:10:03.372008 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.371973    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-trusted-ca-bundle\") pod \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") "
Jul 28 19:10:03.372008 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.372012    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-oauth-config\") pod \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") "
Jul 28 19:10:03.372310 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.372030    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-config\") pod \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") "
Jul 28 19:10:03.372310 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.372065    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-serving-cert\") pod \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") "
Jul 28 19:10:03.372310 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.372085    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-oauth-serving-cert\") pod \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") "
Jul 28 19:10:03.372310 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.372104    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-service-ca\") pod \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") "
Jul 28 19:10:03.372533 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.372412    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "964bd10d-f6f5-4d01-b30f-267a43daa8f4" (UID: "964bd10d-f6f5-4d01-b30f-267a43daa8f4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:10:03.372590 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.372529    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-config" (OuterVolumeSpecName: "console-config") pod "964bd10d-f6f5-4d01-b30f-267a43daa8f4" (UID: "964bd10d-f6f5-4d01-b30f-267a43daa8f4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:10:03.372590 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.372546    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "964bd10d-f6f5-4d01-b30f-267a43daa8f4" (UID: "964bd10d-f6f5-4d01-b30f-267a43daa8f4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:10:03.372590 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.372569    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-service-ca" (OuterVolumeSpecName: "service-ca") pod "964bd10d-f6f5-4d01-b30f-267a43daa8f4" (UID: "964bd10d-f6f5-4d01-b30f-267a43daa8f4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:10:03.374395 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.374367    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "964bd10d-f6f5-4d01-b30f-267a43daa8f4" (UID: "964bd10d-f6f5-4d01-b30f-267a43daa8f4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:10:03.374491 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.374411    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "964bd10d-f6f5-4d01-b30f-267a43daa8f4" (UID: "964bd10d-f6f5-4d01-b30f-267a43daa8f4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:10:03.472688 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.472653    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tffzr\" (UniqueName: \"kubernetes.io/projected/964bd10d-f6f5-4d01-b30f-267a43daa8f4-kube-api-access-tffzr\") pod \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\" (UID: \"964bd10d-f6f5-4d01-b30f-267a43daa8f4\") "
Jul 28 19:10:03.472853 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.472823    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-serving-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:10:03.472853 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.472835    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-oauth-serving-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:10:03.472853 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.472843    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-service-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:10:03.472853 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.472852    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-trusted-ca-bundle\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:10:03.473015 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.472860    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-oauth-config\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:10:03.473015 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.472869    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/964bd10d-f6f5-4d01-b30f-267a43daa8f4-console-config\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:10:03.474876 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.474852    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/964bd10d-f6f5-4d01-b30f-267a43daa8f4-kube-api-access-tffzr" (OuterVolumeSpecName: "kube-api-access-tffzr") pod "964bd10d-f6f5-4d01-b30f-267a43daa8f4" (UID: "964bd10d-f6f5-4d01-b30f-267a43daa8f4"). InnerVolumeSpecName "kube-api-access-tffzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:10:03.573248 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:03.573211    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tffzr\" (UniqueName: \"kubernetes.io/projected/964bd10d-f6f5-4d01-b30f-267a43daa8f4-kube-api-access-tffzr\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:10:04.080865 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:04.080834    2563 generic.go:356] "Generic (PLEG): container finished" podID="964bd10d-f6f5-4d01-b30f-267a43daa8f4" containerID="2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2" exitCode=0
Jul 28 19:10:04.081238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:04.080880    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4cfdbb4f-48z5t"
Jul 28 19:10:04.081238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:04.080938    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4cfdbb4f-48z5t" event={"ID":"964bd10d-f6f5-4d01-b30f-267a43daa8f4","Type":"ContainerDied","Data":"2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2"}
Jul 28 19:10:04.081238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:04.080985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4cfdbb4f-48z5t" event={"ID":"964bd10d-f6f5-4d01-b30f-267a43daa8f4","Type":"ContainerDied","Data":"fb7fec247dc5d8b515016e3f4bf0e7bc62bd04de4104e385630856be2c8556fe"}
Jul 28 19:10:04.081238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:04.081007    2563 scope.go:117] "RemoveContainer" containerID="2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2"
Jul 28 19:10:04.089674 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:04.089657    2563 scope.go:117] "RemoveContainer" containerID="2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2"
Jul 28 19:10:04.089965 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:10:04.089945    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2\": container with ID starting with 2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2 not found: ID does not exist" containerID="2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2"
Jul 28 19:10:04.090033 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:04.089974    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2"} err="failed to get container status \"2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2\": rpc error: code = NotFound desc = could not find container \"2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2\": container with ID starting with 2dd34c2fba4ffc7ab3927a6b7974fb3b934a81a4126fca3588dac24cf52411e2 not found: ID does not exist"
Jul 28 19:10:04.102799 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:04.102776    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d4cfdbb4f-48z5t"]
Jul 28 19:10:04.107319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:04.107298    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d4cfdbb4f-48z5t"]
Jul 28 19:10:05.142740 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:05.142710    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="964bd10d-f6f5-4d01-b30f-267a43daa8f4" path="/var/lib/kubelet/pods/964bd10d-f6f5-4d01-b30f-267a43daa8f4/volumes"
Jul 28 19:10:13.075139 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:13.075097    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:13.255623 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:13.255579    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:15.535058 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.535019    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:15.629698 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.629666    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:15.689662 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.689619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:15.946828 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.946795    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:10:15.947127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.947113    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="964bd10d-f6f5-4d01-b30f-267a43daa8f4" containerName="console"
Jul 28 19:10:15.947181 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.947128    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="964bd10d-f6f5-4d01-b30f-267a43daa8f4" containerName="console"
Jul 28 19:10:15.947217 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.947188    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="964bd10d-f6f5-4d01-b30f-267a43daa8f4" containerName="console"
Jul 28 19:10:15.951631 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.951615    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"
Jul 28 19:10:15.954618 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.954591    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 19:10:15.954748 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.954701    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hz72r\""
Jul 28 19:10:15.955805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.955789    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 19:10:15.955805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.955803    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 19:10:15.960840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.960821    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:10:15.964778 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.964760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/313d514b-2f1c-4680-acc2-77d6de5e14a3-config-file\") pod \"limitador-limitador-54d6d988d5-rx2dw\" (UID: \"313d514b-2f1c-4680-acc2-77d6de5e14a3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"
Jul 28 19:10:15.964877 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.964794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsctm\" (UniqueName: \"kubernetes.io/projected/313d514b-2f1c-4680-acc2-77d6de5e14a3-kube-api-access-fsctm\") pod \"limitador-limitador-54d6d988d5-rx2dw\" (UID: \"313d514b-2f1c-4680-acc2-77d6de5e14a3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"
Jul 28 19:10:15.991627 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:15.991597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:10:16.065572 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:16.065542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/313d514b-2f1c-4680-acc2-77d6de5e14a3-config-file\") pod \"limitador-limitador-54d6d988d5-rx2dw\" (UID: \"313d514b-2f1c-4680-acc2-77d6de5e14a3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"
Jul 28 19:10:16.065763 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:16.065589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsctm\" (UniqueName: \"kubernetes.io/projected/313d514b-2f1c-4680-acc2-77d6de5e14a3-kube-api-access-fsctm\") pod \"limitador-limitador-54d6d988d5-rx2dw\" (UID: \"313d514b-2f1c-4680-acc2-77d6de5e14a3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"
Jul 28 19:10:16.066256 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:16.066235    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/313d514b-2f1c-4680-acc2-77d6de5e14a3-config-file\") pod \"limitador-limitador-54d6d988d5-rx2dw\" (UID: \"313d514b-2f1c-4680-acc2-77d6de5e14a3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"
Jul 28 19:10:16.075020 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:16.074994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsctm\" (UniqueName: \"kubernetes.io/projected/313d514b-2f1c-4680-acc2-77d6de5e14a3-kube-api-access-fsctm\") pod \"limitador-limitador-54d6d988d5-rx2dw\" (UID: \"313d514b-2f1c-4680-acc2-77d6de5e14a3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"
Jul 28 19:10:16.262558 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:16.262465    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"
Jul 28 19:10:16.405152 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:16.404996    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:10:16.408634 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:10:16.408602    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod313d514b_2f1c_4680_acc2_77d6de5e14a3.slice/crio-0b5f0642f92d5a45d73e0ec18734f878cd981a1806d2f3342d5b973c3066cc17 WatchSource:0}: Error finding container 0b5f0642f92d5a45d73e0ec18734f878cd981a1806d2f3342d5b973c3066cc17: Status 404 returned error can't find the container with id 0b5f0642f92d5a45d73e0ec18734f878cd981a1806d2f3342d5b973c3066cc17
Jul 28 19:10:16.448336 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:16.448309    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:16.533266 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:16.533174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:17.148506 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:17.148454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw" event={"ID":"313d514b-2f1c-4680-acc2-77d6de5e14a3","Type":"ContainerStarted","Data":"0b5f0642f92d5a45d73e0ec18734f878cd981a1806d2f3342d5b973c3066cc17"}
Jul 28 19:10:19.151176 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:19.151142    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw" event={"ID":"313d514b-2f1c-4680-acc2-77d6de5e14a3","Type":"ContainerStarted","Data":"6a081b6edca7cb23559dcc0a10b2065a2c16c51d26bed62d00454dc226aa63ea"}
Jul 28 19:10:19.151568 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:19.151266    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"
Jul 28 19:10:19.168898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:19.168799    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw" podStartSLOduration=1.6649781510000001 podStartE2EDuration="4.168785509s" totalImagesPullingTime="2.503807358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:10:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:10:19.167985481 +0000 UTC m=+460.629593185" watchObservedRunningTime="2026-07-28 19:10:19.168785509 +0000 UTC m=+460.630393211"
Jul 28 19:10:30.155756 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:30.155732    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"
Jul 28 19:10:31.637977 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:31.637946    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:31.735533 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:31.735505    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:32.155156 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:32.155119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:43.975350 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:43.975310    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:44.118928 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:44.118899    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:50.278326 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:50.278286    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:50.390745 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:50.390699    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:50.490198 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:50.490172    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:53.821116 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:53.821068    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:10:53.921763 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:10:53.921734    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:24.838298 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:24.838268    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:25.023206 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:25.023174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:25.515805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:25.515776    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:25.918710 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:25.918659    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:26.389829 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:26.389789    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:26.504284 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:26.504252    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:26.594573 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:26.594539    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:11:36.684615 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:36.684534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:36.788361 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:36.788325    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:39.766277 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:39.766245    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:40.975697 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:40.975660    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:41.570152 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:41.570117    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:11:56.680778 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:56.680740    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:11:57.361325 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:11:57.361287    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:12:06.993463 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:06.993431    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:12:08.162893 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:08.162856    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:12:08.617068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:08.617032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:12:08.710855 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:08.710814    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:12:25.441250 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.441218    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"]
Jul 28 19:12:25.444436 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.444415    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.448381 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.448359    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-twk7j\""
Jul 28 19:12:25.448519 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.448392    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 19:12:25.448519 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.448407    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 19:12:25.448519 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.448392    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 19:12:25.454532 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.454510    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"]
Jul 28 19:12:25.476007 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.475983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:12:25.606096 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.606043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.606096 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.606093    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtlx9\" (UniqueName: \"kubernetes.io/projected/7ce32be4-d620-4644-8db4-6d96ea24f069-kube-api-access-dtlx9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.606322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.606149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.606322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.606204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ce32be4-d620-4644-8db4-6d96ea24f069-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.606322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.606232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.606322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.606255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.707545 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.707456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.707545 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.707494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtlx9\" (UniqueName: \"kubernetes.io/projected/7ce32be4-d620-4644-8db4-6d96ea24f069-kube-api-access-dtlx9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.707545 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.707517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.707788 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.707549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ce32be4-d620-4644-8db4-6d96ea24f069-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.707788 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.707574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.707788 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.707591    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.707984 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.707960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.708083 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.707986    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.708083 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.708027    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.709966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.709947    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ce32be4-d620-4644-8db4-6d96ea24f069-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.710231 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.710215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ce32be4-d620-4644-8db4-6d96ea24f069-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.716581 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.716548    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtlx9\" (UniqueName: \"kubernetes.io/projected/7ce32be4-d620-4644-8db4-6d96ea24f069-kube-api-access-dtlx9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89\" (UID: \"7ce32be4-d620-4644-8db4-6d96ea24f069\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.755434 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.755407    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:25.890598 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:25.890566    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"]
Jul 28 19:12:25.893594 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:12:25.893556    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce32be4_d620_4644_8db4_6d96ea24f069.slice/crio-902028185f4aa548c532bba3773d68e89b7d1880e0a8b55cf1c6b8b738aec7c8 WatchSource:0}: Error finding container 902028185f4aa548c532bba3773d68e89b7d1880e0a8b55cf1c6b8b738aec7c8: Status 404 returned error can't find the container with id 902028185f4aa548c532bba3773d68e89b7d1880e0a8b55cf1c6b8b738aec7c8
Jul 28 19:12:26.682604 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:26.682546    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89" event={"ID":"7ce32be4-d620-4644-8db4-6d96ea24f069","Type":"ContainerStarted","Data":"902028185f4aa548c532bba3773d68e89b7d1880e0a8b55cf1c6b8b738aec7c8"}
Jul 28 19:12:27.048740 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:27.048652    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:12:27.688295 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:27.688258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89" event={"ID":"7ce32be4-d620-4644-8db4-6d96ea24f069","Type":"ContainerStarted","Data":"13df2cb02c4ccdf935552957d0e3c99a41460fe04ff0194789e6d4ddc73aff98"}
Jul 28 19:12:27.710338 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:27.710277    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89" podStartSLOduration=1.74910007 podStartE2EDuration="2.710260246s" totalImagesPullingTime="961.160176ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:12:27.708482256 +0000 UTC m=+589.170089961" watchObservedRunningTime="2026-07-28 19:12:27.710260246 +0000 UTC m=+589.171867949"
Jul 28 19:12:32.076764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:32.076721    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:12:34.665476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:34.665438    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:12:35.755988 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:35.755945    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:35.756418 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:35.756015    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:35.768809 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:35.768782    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:36.735955 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:36.735906    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89"
Jul 28 19:12:38.963546 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:38.963519    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:12:38.963976 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:38.963705    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:12:41.975480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:41.975448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:12:48.082539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:48.082507    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:12:50.364623 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:12:50.364587    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:13:04.451082 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:13:04.451002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:13:57.865554 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:13:57.865500    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:13:58.034044 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:13:58.034014    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:13:58.979118 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:13:58.979082    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:13:59.084741 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:13:59.084707    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:13:59.898148 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:13:59.898116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:14:05.753122 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:14:05.753083    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:14:21.871701 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:14:21.871660    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:14:21.973170 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:14:21.973137    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:14:27.638560 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:14:27.638526    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:14:38.674302 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:14:38.674221    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:14:49.088056 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:14:49.088014    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:15:11.779584 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:15:11.779546    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:15:11.866867 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:15:11.866839    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:16:05.072978 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:05.072863    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:16:06.319298 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:06.319263    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:16:27.887605 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:27.887569    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:16:42.298868 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:42.298835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:16:49.918361 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:49.918314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:16:50.029720 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:50.029669    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:16:50.132992 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:50.132962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:16:57.237761 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:57.237711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:16:57.426825 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:57.426795    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:16:57.651496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:57.651461    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:16:57.835431 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:16:57.835386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:17:04.255095 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:17:04.255062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:17:04.356410 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:17:04.356380    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:17:22.373673 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:17:22.373638    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:17:31.577187 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:17:31.577102    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:17:38.987449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:17:38.987421    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:17:38.988465 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:17:38.988440    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:17:41.675130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:17:41.675091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:17:49.980168 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:17:49.980131    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:18:02.178792 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:18:02.178754    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:18:11.282479 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:18:11.282444    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:18:21.873600 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:18:21.873562    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:18:31.062981 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:18:31.062946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:18:54.071336 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:18:54.071286    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:19:03.664298 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:19:03.664214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:19:28.078766 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:19:28.078730    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:19:39.073830 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:19:39.073787    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:19:52.572377 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:19:52.572342    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:20:06.870796 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:20:06.870759    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:20:18.073671 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:20:18.073636    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:20:27.366825 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:20:27.366793    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:20:27.601071 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:20:27.601030    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:20:36.464236 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:20:36.464158    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:20:45.482419 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:20:45.482375    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:21:58.914633 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:21:58.914597    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:22:34.257144 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:22:34.257108    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:22:39.009956 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:22:39.009907    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:22:39.012221 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:22:39.012202    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:22:43.201095 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:22:43.201057    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:22:51.848728 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:22:51.848672    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:22:51.921096 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:22:51.921062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:22:59.357962 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:22:59.357912    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:23:09.253091 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:23:09.253060    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:23:18.647874 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:23:18.647839    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:23:26.253708 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:23:26.253670    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:23:36.850902 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:23:36.850806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:23:49.051696 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:23:49.051657    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:23:57.957405 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:23:57.957367    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:24:07.025426 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:24:07.025393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:24:08.762663 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:24:08.762626    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:24:21.355498 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:24:21.355461    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:24:31.754040 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:24:31.754004    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:24:44.958795 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:24:44.958765    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:24:55.146203 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:24:55.146167    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:25:16.806804 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:25:16.806723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:25:20.053302 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:25:20.053268    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:25:27.054026 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:25:27.053985    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:25:39.247952 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:25:39.246685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:26:01.562735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:26:01.562700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:26:02.952367 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:26:02.952337    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:26:05.354821 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:26:05.354786    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:26:18.448410 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:26:18.448373    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:26:27.643586 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:26:27.643548    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:26:39.047295 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:26:39.047255    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:26:48.748420 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:26:48.748378    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:26:57.750067 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:26:57.750031    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:27:07.246992 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:27:07.246953    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:27:18.348311 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:27:18.348277    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:27:27.847913 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:27:27.847877    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:27:39.032029 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:27:39.031998    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:27:39.038090 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:27:39.038070    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:27:40.447358 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:27:40.447319    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:27:49.274977 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:27:49.274935    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:27:58.877586 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:27:58.877545    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:28:10.981793 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:28:10.981758    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:28:22.854189 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:28:22.854151    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:28:31.674137 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:28:31.674098    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:28:35.906259 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:28:35.906222    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:28:42.951331 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:28:42.951286    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:28:55.145463 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:28:55.145429    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:29:03.846761 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:03.846725    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:29:13.849414 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:13.849378    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:29:16.657619 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:16.657577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:22.846613 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:22.846577    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:29:28.205841 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:28.205806    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:40.428966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:40.428867    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:43.493185 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:43.493148    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:43.597966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:43.597908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:43.995110 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:43.995071    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:44.355934 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:44.355889    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:29:49.879703 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:49.879672    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:49.950136 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:49.950052    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:50.116529 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.116497    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:50.374313 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.374284    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"]
Jul 28 19:29:50.377441 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.377426    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.380328 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.380304    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-16c9dd30-openshift-default-dockercfg-pnh7q\""
Jul 28 19:29:50.400774 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.400738    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"]
Jul 28 19:29:50.413186 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.413161    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:50.547285 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.547242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-certs\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.547495 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.547328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-podinfo\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.547495 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.547355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-data\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.547495 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.547389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c76d830-9da1-443c-9972-4ed5a35450cf-istiod-ca-cert\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.547495 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.547416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgq2r\" (UniqueName: \"kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-kube-api-access-jgq2r\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.547495 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.547452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-envoy\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.547695 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.547521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-socket\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.547695 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.547552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-credential-socket\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.547695 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.547568    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-token\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.648576 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.648478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-socket\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.648576 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.648523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-credential-socket\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.648576 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.648548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-token\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.648840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.648584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-certs\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.648840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.648641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-podinfo\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.648840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.648665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-data\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.648840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.648703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c76d830-9da1-443c-9972-4ed5a35450cf-istiod-ca-cert\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.648840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.648734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgq2r\" (UniqueName: \"kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-kube-api-access-jgq2r\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.648840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.648775    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-envoy\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.649098 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.649026    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-credential-socket\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.649098 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.649030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-socket\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.649615 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.649585    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c76d830-9da1-443c-9972-4ed5a35450cf-istiod-ca-cert\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.649742 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.649643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-data\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.649742 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.649662    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-certs\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.651527 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.651498    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-envoy\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.651645 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.651537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-podinfo\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.658059 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.658031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-token\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.658326 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.658295    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgq2r\" (UniqueName: \"kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-kube-api-access-jgq2r\") pod \"e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") " pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.688167 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.688126    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:50.823661 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.823504    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"]
Jul 28 19:29:50.826346 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:29:50.826316    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c76d830_9da1_443c_9972_4ed5a35450cf.slice/crio-f53ccc57846b13fba17bb023629672fda45da359dba99915d47aa2705233a088 WatchSource:0}: Error finding container f53ccc57846b13fba17bb023629672fda45da359dba99915d47aa2705233a088: Status 404 returned error can't find the container with id f53ccc57846b13fba17bb023629672fda45da359dba99915d47aa2705233a088
Jul 28 19:29:50.828596 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.828568    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:29:50.828668 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.828636    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:29:50.828722 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.828663    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:29:50.940172 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.940082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" event={"ID":"8c76d830-9da1-443c-9972-4ed5a35450cf","Type":"ContainerStarted","Data":"3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565"}
Jul 28 19:29:50.940172 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.940144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" event={"ID":"8c76d830-9da1-443c-9972-4ed5a35450cf","Type":"ContainerStarted","Data":"f53ccc57846b13fba17bb023629672fda45da359dba99915d47aa2705233a088"}
Jul 28 19:29:50.963687 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:50.963635    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" podStartSLOduration=0.963615918 podStartE2EDuration="963.615918ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:29:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:50.962264572 +0000 UTC m=+1632.423872275" watchObservedRunningTime="2026-07-28 19:29:50.963615918 +0000 UTC m=+1632.425223621"
Jul 28 19:29:51.332998 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:51.332966    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:51.688728 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:51.688643    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:51.693374 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:51.693354    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:51.944649 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:51.944569    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:51.945484 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:51.945468    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:29:52.319362 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:52.319336    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:52.415697 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:52.415657    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:52.515425 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:52.515397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:52.625289 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:52.625188    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:53.096669 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:53.096636    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:53.196783 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:53.196749    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:29:59.943943 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:29:59.943890    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:30:07.987078 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:07.987036    2563 patch_prober.go:28] interesting pod/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 19:30:07.988856 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:07.987102    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 19:30:09.015330 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:09.015290    2563 generic.go:356] "Generic (PLEG): container finished" podID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerID="3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565" exitCode=0
Jul 28 19:30:09.015330 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:09.015337    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" event={"ID":"8c76d830-9da1-443c-9972-4ed5a35450cf","Type":"ContainerDied","Data":"3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565"}
Jul 28 19:30:09.015745 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:09.015688    2563 scope.go:117] "RemoveContainer" containerID="3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565"
Jul 28 19:30:09.016607 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:09.016582    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:30:09.016667 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:09.016654    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:30:09.016701 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:09.016688    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:30:10.023050 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:10.023017    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" event={"ID":"8c76d830-9da1-443c-9972-4ed5a35450cf","Type":"ContainerStarted","Data":"e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448"}
Jul 28 19:30:10.689134 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:10.689099    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:30:10.689339 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:10.689317    2563 patch_prober.go:28] interesting pod/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 28 19:30:10.689394 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:10.689371    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 28 19:30:11.688912 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:11.688871    2563 patch_prober.go:28] interesting pod/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 28 19:30:11.689331 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:11.688971    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 28 19:30:12.689059 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:12.689022    2563 patch_prober.go:28] interesting pod/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 28 19:30:12.689438 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:12.689078    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 28 19:30:13.689230 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:13.689190    2563 patch_prober.go:28] interesting pod/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 28 19:30:13.689627 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:13.689251    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 28 19:30:14.692126 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:14.692093    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:30:14.692501 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:14.692472    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:30:14.693110 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:14.693091    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:30:17.863685 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:17.863641    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"]
Jul 28 19:30:17.866383 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:17.866361    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"
Jul 28 19:30:17.869399 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:17.869369    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:30:17.869533 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:17.869408    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:30:17.875069 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:17.875043    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"]
Jul 28 19:30:17.987492 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:17.987454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d72f637-a886-4bdf-bd61-b36470a4cc40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz\" (UID: \"7d72f637-a886-4bdf-bd61-b36470a4cc40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"
Jul 28 19:30:18.088415 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:18.088375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d72f637-a886-4bdf-bd61-b36470a4cc40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz\" (UID: \"7d72f637-a886-4bdf-bd61-b36470a4cc40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"
Jul 28 19:30:18.089091 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:18.089072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d72f637-a886-4bdf-bd61-b36470a4cc40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz\" (UID: \"7d72f637-a886-4bdf-bd61-b36470a4cc40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"
Jul 28 19:30:18.177866 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:18.177787    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"
Jul 28 19:30:18.313172 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:18.313145    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"]
Jul 28 19:30:18.315418 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:30:18.315393    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d72f637_a886_4bdf_bd61_b36470a4cc40.slice/crio-2bfd24bf2ab2865ef23fd3ca4d36c3a233175fe6d675d224c1edfbf9f8d72c33 WatchSource:0}: Error finding container 2bfd24bf2ab2865ef23fd3ca4d36c3a233175fe6d675d224c1edfbf9f8d72c33: Status 404 returned error can't find the container with id 2bfd24bf2ab2865ef23fd3ca4d36c3a233175fe6d675d224c1edfbf9f8d72c33
Jul 28 19:30:18.317294 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:18.317274    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:30:19.063978 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:19.063947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz" event={"ID":"7d72f637-a886-4bdf-bd61-b36470a4cc40","Type":"ContainerStarted","Data":"2bfd24bf2ab2865ef23fd3ca4d36c3a233175fe6d675d224c1edfbf9f8d72c33"}
Jul 28 19:30:22.078274 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:22.078233    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d72f637-a886-4bdf-bd61-b36470a4cc40" containerID="02c5c05810d1d18d2e4255e764ac62a20f45ee7040123b3a46bdf04b2adea668" exitCode=0
Jul 28 19:30:22.078659 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:22.078300    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz" event={"ID":"7d72f637-a886-4bdf-bd61-b36470a4cc40","Type":"ContainerDied","Data":"02c5c05810d1d18d2e4255e764ac62a20f45ee7040123b3a46bdf04b2adea668"}
Jul 28 19:30:23.212767 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:23.212743    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"
Jul 28 19:30:23.332129 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:23.332096    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d72f637-a886-4bdf-bd61-b36470a4cc40-maas-api-service-ca\") pod \"7d72f637-a886-4bdf-bd61-b36470a4cc40\" (UID: \"7d72f637-a886-4bdf-bd61-b36470a4cc40\") "
Jul 28 19:30:23.332463 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:23.332440    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d72f637-a886-4bdf-bd61-b36470a4cc40-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7d72f637-a886-4bdf-bd61-b36470a4cc40" (UID: "7d72f637-a886-4bdf-bd61-b36470a4cc40"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:30:23.433586 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:23.433507    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d72f637-a886-4bdf-bd61-b36470a4cc40-maas-api-service-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:30:24.088107 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:24.088076    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"
Jul 28 19:30:24.088272 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:24.088079    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz" event={"ID":"7d72f637-a886-4bdf-bd61-b36470a4cc40","Type":"ContainerDied","Data":"2bfd24bf2ab2865ef23fd3ca4d36c3a233175fe6d675d224c1edfbf9f8d72c33"}
Jul 28 19:30:24.088272 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:24.088199    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bfd24bf2ab2865ef23fd3ca4d36c3a233175fe6d675d224c1edfbf9f8d72c33"
Jul 28 19:30:33.195939 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:33.195882    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:33.296453 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:33.296410    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:33.372347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:33.372319    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"]
Jul 28 19:30:33.375396 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:33.375361    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-16c9dd30-qbzrz"]
Jul 28 19:30:33.375553 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:33.375530    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:33.438075 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:33.438031    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:33.520397 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:33.520311    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"]
Jul 28 19:30:33.520602 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:33.520572    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy" containerID="cri-o://e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448" gracePeriod=30
Jul 28 19:30:33.540156 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:33.540126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:33.781457 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:33.781369    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:34.011537 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.011501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:34.111530 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.111501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:34.208758 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.208726    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:34.511152 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.511065    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"]
Jul 28 19:30:34.511414 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.511397    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d72f637-a886-4bdf-bd61-b36470a4cc40" containerName="revoke-keys"
Jul 28 19:30:34.511480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.511417    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d72f637-a886-4bdf-bd61-b36470a4cc40" containerName="revoke-keys"
Jul 28 19:30:34.511538 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.511525    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d72f637-a886-4bdf-bd61-b36470a4cc40" containerName="revoke-keys"
Jul 28 19:30:34.513610 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.513591    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.516433 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.516417    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-84845342-gw-openshift-default-dockercfg-s78v7\""
Jul 28 19:30:34.525822 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.525795    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"]
Jul 28 19:30:34.539655 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.539624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:34.629639 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.629597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-socket\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.629639 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.629640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-podinfo\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.629841 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.629670    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-credential-socket\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.629841 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.629776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-envoy\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.629841 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.629832    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-data\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.629991 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.629875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-token\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.629991 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.629895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-certs\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.629991 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.629956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istiod-ca-cert\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.629991 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.629972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9z24\" (UniqueName: \"kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-kube-api-access-n9z24\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.656208 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.656162    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:34.730404 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.730368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-podinfo\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.730580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.730419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-credential-socket\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.730580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.730463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-envoy\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.730580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.730501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-data\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.730580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.730534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-token\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.730580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.730557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-certs\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.730824 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.730656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istiod-ca-cert\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.730824 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.730690    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9z24\" (UniqueName: \"kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-kube-api-access-n9z24\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.730824 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.730747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-socket\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.731097 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.731074    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-data\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.731275 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.731133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-socket\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.731411 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.731320    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-credential-socket\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.731411 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.731356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-certs\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.731518 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.731422    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istiod-ca-cert\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.733008 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.732987    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-envoy\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.733173 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.733157    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-podinfo\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.739611 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.739585    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-token\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.739707 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.739645    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9z24\" (UniqueName: \"kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-kube-api-access-n9z24\") pod \"e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") " pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.824383 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.824352    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:34.963112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.963086    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"]
Jul 28 19:30:34.965174 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:30:34.965141    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17fedbd_2a41_42bb_91a0_76fa6410df0c.slice/crio-d0809c6f9ba530e054af37dc97a3e7ffc613b00a239cdb823eb2b46ea648d37f WatchSource:0}: Error finding container d0809c6f9ba530e054af37dc97a3e7ffc613b00a239cdb823eb2b46ea648d37f: Status 404 returned error can't find the container with id d0809c6f9ba530e054af37dc97a3e7ffc613b00a239cdb823eb2b46ea648d37f
Jul 28 19:30:34.967214 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.967182    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:30:34.967288 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.967251    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:30:34.967288 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:34.967280    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:30:35.136778 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:35.136692    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" event={"ID":"c17fedbd-2a41-42bb-91a0-76fa6410df0c","Type":"ContainerStarted","Data":"dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f"}
Jul 28 19:30:35.136778 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:35.136754    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" event={"ID":"c17fedbd-2a41-42bb-91a0-76fa6410df0c","Type":"ContainerStarted","Data":"d0809c6f9ba530e054af37dc97a3e7ffc613b00a239cdb823eb2b46ea648d37f"}
Jul 28 19:30:35.143664 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:35.143622    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d72f637-a886-4bdf-bd61-b36470a4cc40" path="/var/lib/kubelet/pods/7d72f637-a886-4bdf-bd61-b36470a4cc40/volumes"
Jul 28 19:30:35.157640 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:35.157592    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" podStartSLOduration=1.157574537 podStartE2EDuration="1.157574537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:30:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:30:35.156053877 +0000 UTC m=+1676.617661578" watchObservedRunningTime="2026-07-28 19:30:35.157574537 +0000 UTC m=+1676.619182239"
Jul 28 19:30:35.824699 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:35.824665    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:36.629379 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:36.629348    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:36.751258 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:36.751223    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:36.830199 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:36.830171    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:36.972291 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:36.972190    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:37.145138 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:37.145109    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:37.146204 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:37.146188    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:37.252022 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:37.251934    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:37.349388 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:37.349355    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:30:37.551024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:37.550933    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:30:38.765265 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.765240    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:30:38.869401 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869352    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-podinfo\") pod \"8c76d830-9da1-443c-9972-4ed5a35450cf\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") "
Jul 28 19:30:38.869401 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869405    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-data\") pod \"8c76d830-9da1-443c-9972-4ed5a35450cf\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") "
Jul 28 19:30:38.869648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869425    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-certs\") pod \"8c76d830-9da1-443c-9972-4ed5a35450cf\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") "
Jul 28 19:30:38.869648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869443    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c76d830-9da1-443c-9972-4ed5a35450cf-istiod-ca-cert\") pod \"8c76d830-9da1-443c-9972-4ed5a35450cf\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") "
Jul 28 19:30:38.869648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869487    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-envoy\") pod \"8c76d830-9da1-443c-9972-4ed5a35450cf\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") "
Jul 28 19:30:38.869648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869505    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-socket\") pod \"8c76d830-9da1-443c-9972-4ed5a35450cf\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") "
Jul 28 19:30:38.869648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869539    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-credential-socket\") pod \"8c76d830-9da1-443c-9972-4ed5a35450cf\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") "
Jul 28 19:30:38.869648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869597    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgq2r\" (UniqueName: \"kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-kube-api-access-jgq2r\") pod \"8c76d830-9da1-443c-9972-4ed5a35450cf\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") "
Jul 28 19:30:38.869648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869630    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-token\") pod \"8c76d830-9da1-443c-9972-4ed5a35450cf\" (UID: \"8c76d830-9da1-443c-9972-4ed5a35450cf\") "
Jul 28 19:30:38.870053 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869687    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c76d830-9da1-443c-9972-4ed5a35450cf" (UID: "8c76d830-9da1-443c-9972-4ed5a35450cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:30:38.870053 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869890    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c76d830-9da1-443c-9972-4ed5a35450cf" (UID: "8c76d830-9da1-443c-9972-4ed5a35450cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:30:38.870053 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.869905    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:30:38.870053 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.870013    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c76d830-9da1-443c-9972-4ed5a35450cf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c76d830-9da1-443c-9972-4ed5a35450cf" (UID: "8c76d830-9da1-443c-9972-4ed5a35450cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:30:38.870053 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.870038    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c76d830-9da1-443c-9972-4ed5a35450cf" (UID: "8c76d830-9da1-443c-9972-4ed5a35450cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:30:38.870661 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.870636    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c76d830-9da1-443c-9972-4ed5a35450cf" (UID: "8c76d830-9da1-443c-9972-4ed5a35450cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:30:38.872110 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.872082    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c76d830-9da1-443c-9972-4ed5a35450cf" (UID: "8c76d830-9da1-443c-9972-4ed5a35450cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:30:38.872254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.872230    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-kube-api-access-jgq2r" (OuterVolumeSpecName: "kube-api-access-jgq2r") pod "8c76d830-9da1-443c-9972-4ed5a35450cf" (UID: "8c76d830-9da1-443c-9972-4ed5a35450cf"). InnerVolumeSpecName "kube-api-access-jgq2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:30:38.872345 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.872330    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c76d830-9da1-443c-9972-4ed5a35450cf" (UID: "8c76d830-9da1-443c-9972-4ed5a35450cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:30:38.872394 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.872372    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c76d830-9da1-443c-9972-4ed5a35450cf" (UID: "8c76d830-9da1-443c-9972-4ed5a35450cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:30:38.970743 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.970707    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:30:38.970743 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.970734    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:30:38.970743 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.970743    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:30:38.970743 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.970755    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgq2r\" (UniqueName: \"kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-kube-api-access-jgq2r\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:30:38.971031 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.970764    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:30:38.971031 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.970772    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:30:38.971031 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.970779    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c76d830-9da1-443c-9972-4ed5a35450cf-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:30:38.971031 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:38.970786    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c76d830-9da1-443c-9972-4ed5a35450cf-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:30:39.155091 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.155066    2563 generic.go:356] "Generic (PLEG): container finished" podID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerID="e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448" exitCode=0
Jul 28 19:30:39.155220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.155120    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"
Jul 28 19:30:39.155220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.155144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" event={"ID":"8c76d830-9da1-443c-9972-4ed5a35450cf","Type":"ContainerDied","Data":"e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448"}
Jul 28 19:30:39.155220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.155205    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm" event={"ID":"8c76d830-9da1-443c-9972-4ed5a35450cf","Type":"ContainerDied","Data":"f53ccc57846b13fba17bb023629672fda45da359dba99915d47aa2705233a088"}
Jul 28 19:30:39.155323 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.155236    2563 scope.go:117] "RemoveContainer" containerID="e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448"
Jul 28 19:30:39.164293 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.164273    2563 scope.go:117] "RemoveContainer" containerID="3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565"
Jul 28 19:30:39.172306 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.172286    2563 scope.go:117] "RemoveContainer" containerID="e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448"
Jul 28 19:30:39.172630 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:30:39.172613    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448\": container with ID starting with e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448 not found: ID does not exist" containerID="e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448"
Jul 28 19:30:39.172702 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.172644    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448"} err="failed to get container status \"e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448\": rpc error: code = NotFound desc = could not find container \"e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448\": container with ID starting with e8a2ff617df821786072570d7f23bb0966167164d71a2ec51c2bfdfb0f615448 not found: ID does not exist"
Jul 28 19:30:39.172702 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.172669    2563 scope.go:117] "RemoveContainer" containerID="3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565"
Jul 28 19:30:39.172910 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:30:39.172893    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565\": container with ID starting with 3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565 not found: ID does not exist" containerID="3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565"
Jul 28 19:30:39.172992 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.172935    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565"} err="failed to get container status \"3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565\": rpc error: code = NotFound desc = could not find container \"3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565\": container with ID starting with 3f392299f0fcf71db88cc1a679120bc0314c76b4e06180064ee65f2bdbce0565 not found: ID does not exist"
Jul 28 19:30:39.176365 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.176341    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"]
Jul 28 19:30:39.178580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:39.178560    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-16c9dd30-openshift-default-77d99f4d8-8qwjm"]
Jul 28 19:30:41.143253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:41.143216    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" path="/var/lib/kubelet/pods/8c76d830-9da1-443c-9972-4ed5a35450cf/volumes"
Jul 28 19:30:47.951338 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:47.951300    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:30:51.212054 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:51.212020    2563 generic.go:356] "Generic (PLEG): container finished" podID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerID="dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f" exitCode=0
Jul 28 19:30:51.212489 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:51.212068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" event={"ID":"c17fedbd-2a41-42bb-91a0-76fa6410df0c","Type":"ContainerDied","Data":"dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f"}
Jul 28 19:30:51.212489 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:51.212414    2563 scope.go:117] "RemoveContainer" containerID="dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f"
Jul 28 19:30:51.213424 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:51.213398    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:30:51.213530 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:51.213479    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:30:51.213650 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:51.213525    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:30:52.145683 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:52.145643    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:52.219840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:52.219800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" event={"ID":"c17fedbd-2a41-42bb-91a0-76fa6410df0c","Type":"ContainerStarted","Data":"142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0"}
Jul 28 19:30:52.824836 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:52.824789    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:52.825953 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:52.825901    2563 patch_prober.go:28] interesting pod/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 28 19:30:52.826075 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:52.825977    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 28 19:30:53.825244 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:53.825209    2563 patch_prober.go:28] interesting pod/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 28 19:30:53.825642 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:53.825266    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 28 19:30:54.825586 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:54.825548    2563 patch_prober.go:28] interesting pod/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 28 19:30:54.825980 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:54.825604    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 28 19:30:55.825808 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:55.825762    2563 patch_prober.go:28] interesting pod/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 28 19:30:55.826233 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:55.825833    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 28 19:30:56.828301 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:56.828264    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:56.828666 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:56.828524    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:30:56.829298 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:30:56.829283    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:31:01.617296 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.617261    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"]
Jul 28 19:31:01.617654 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.617549    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy"
Jul 28 19:31:01.617654 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.617559    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy"
Jul 28 19:31:01.617654 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.617569    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy"
Jul 28 19:31:01.617654 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.617575    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy"
Jul 28 19:31:01.617654 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.617626    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy"
Jul 28 19:31:01.617654 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.617635    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c76d830-9da1-443c-9972-4ed5a35450cf" containerName="istio-proxy"
Jul 28 19:31:01.621508 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.621492    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"
Jul 28 19:31:01.624066 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.624044    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:31:01.624193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.624069    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:31:01.627348 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.627324    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"]
Jul 28 19:31:01.764784 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.764749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b04db743-70f2-4881-bb86-ba5c84077562-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-84845342-m9b96\" (UID: \"b04db743-70f2-4881-bb86-ba5c84077562\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"
Jul 28 19:31:01.866219 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.866183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b04db743-70f2-4881-bb86-ba5c84077562-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-84845342-m9b96\" (UID: \"b04db743-70f2-4881-bb86-ba5c84077562\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"
Jul 28 19:31:01.866880 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.866851    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b04db743-70f2-4881-bb86-ba5c84077562-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-84845342-m9b96\" (UID: \"b04db743-70f2-4881-bb86-ba5c84077562\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"
Jul 28 19:31:01.933085 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:01.933003    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"
Jul 28 19:31:02.064712 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:02.064615    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"]
Jul 28 19:31:02.066985 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:31:02.066958    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb04db743_70f2_4881_bb86_ba5c84077562.slice/crio-949facb1f2ff2ffc9c8f9cfc63e4c750b866d8709f42bdcdf6fe660ab6a360c7 WatchSource:0}: Error finding container 949facb1f2ff2ffc9c8f9cfc63e4c750b866d8709f42bdcdf6fe660ab6a360c7: Status 404 returned error can't find the container with id 949facb1f2ff2ffc9c8f9cfc63e4c750b866d8709f42bdcdf6fe660ab6a360c7
Jul 28 19:31:02.265457 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:02.265373    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96" event={"ID":"b04db743-70f2-4881-bb86-ba5c84077562","Type":"ContainerStarted","Data":"949facb1f2ff2ffc9c8f9cfc63e4c750b866d8709f42bdcdf6fe660ab6a360c7"}
Jul 28 19:31:03.270735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:03.270701    2563 generic.go:356] "Generic (PLEG): container finished" podID="b04db743-70f2-4881-bb86-ba5c84077562" containerID="2bc2507bbf02eefb2ba0163f767e8b81dabd7185d4384e62e04f1e6717f4b480" exitCode=0
Jul 28 19:31:03.271150 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:03.270750    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96" event={"ID":"b04db743-70f2-4881-bb86-ba5c84077562","Type":"ContainerDied","Data":"2bc2507bbf02eefb2ba0163f767e8b81dabd7185d4384e62e04f1e6717f4b480"}
Jul 28 19:31:04.404048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:04.404017    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"
Jul 28 19:31:04.487944 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:04.487884    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b04db743-70f2-4881-bb86-ba5c84077562-maas-api-service-ca\") pod \"b04db743-70f2-4881-bb86-ba5c84077562\" (UID: \"b04db743-70f2-4881-bb86-ba5c84077562\") "
Jul 28 19:31:04.488256 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:04.488231    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b04db743-70f2-4881-bb86-ba5c84077562-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b04db743-70f2-4881-bb86-ba5c84077562" (UID: "b04db743-70f2-4881-bb86-ba5c84077562"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:31:04.588574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:04.588540    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b04db743-70f2-4881-bb86-ba5c84077562-maas-api-service-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:05.280459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:05.280435    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"
Jul 28 19:31:05.280648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:05.280433    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96" event={"ID":"b04db743-70f2-4881-bb86-ba5c84077562","Type":"ContainerDied","Data":"949facb1f2ff2ffc9c8f9cfc63e4c750b866d8709f42bdcdf6fe660ab6a360c7"}
Jul 28 19:31:05.280648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:05.280559    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="949facb1f2ff2ffc9c8f9cfc63e4c750b866d8709f42bdcdf6fe660ab6a360c7"
Jul 28 19:31:08.314771 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:08.314739    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:08.444438 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:08.444398    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"]
Jul 28 19:31:08.450147 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:08.450104    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-84845342-m9b96"]
Jul 28 19:31:08.601283 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:08.601252    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"]
Jul 28 19:31:08.601523 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:08.601481    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy" containerID="cri-o://142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0" gracePeriod=30
Jul 28 19:31:09.143348 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.143314    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b04db743-70f2-4881-bb86-ba5c84077562" path="/var/lib/kubelet/pods/b04db743-70f2-4881-bb86-ba5c84077562/volumes"
Jul 28 19:31:09.372965 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.372905    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"]
Jul 28 19:31:09.373327 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.373243    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b04db743-70f2-4881-bb86-ba5c84077562" containerName="revoke-keys"
Jul 28 19:31:09.373327 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.373255    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b04db743-70f2-4881-bb86-ba5c84077562" containerName="revoke-keys"
Jul 28 19:31:09.373327 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.373326    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b04db743-70f2-4881-bb86-ba5c84077562" containerName="revoke-keys"
Jul 28 19:31:09.376592 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.376570    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.379387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.379364    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-43235862-openshift-default-dockercfg-nns42\""
Jul 28 19:31:09.392558 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.392517    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"]
Jul 28 19:31:09.421426 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.421335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-token\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.421426 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.421383    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-envoy\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.421426 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.421415    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-socket\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.421671 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.421491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-data\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.421671 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.421576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-certs\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.421671 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.421612    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-credential-socket\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.421671 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.421638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-podinfo\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.421671 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.421664    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1794678e-72dd-47b2-ae07-d349a51f93a3-istiod-ca-cert\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.421822 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.421699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7tzx\" (UniqueName: \"kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-kube-api-access-s7tzx\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.522777 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.522737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7tzx\" (UniqueName: \"kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-kube-api-access-s7tzx\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.522981 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.522789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-token\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.522981 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.522815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-envoy\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.522981 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.522834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-socket\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.523125 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.523042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-data\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.523181 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.523127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-certs\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.523181 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.523167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-credential-socket\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.523359 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.523193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-podinfo\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.523359 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.523214    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-socket\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.523359 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.523217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1794678e-72dd-47b2-ae07-d349a51f93a3-istiod-ca-cert\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.523520 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.523437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-data\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.523520 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.523494    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-certs\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.523756 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.523603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-credential-socket\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.523756 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.523744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1794678e-72dd-47b2-ae07-d349a51f93a3-istiod-ca-cert\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.525247 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.525228    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-envoy\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.525754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.525731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-podinfo\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.534271 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.534241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-token\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.534437 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.534420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7tzx\" (UniqueName: \"kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-kube-api-access-s7tzx\") pod \"e2e-ait-43235862-openshift-default-84c7dd5878-w82m9\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") " pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.687303 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.687212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:09.690115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.690072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:09.832141 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.832113    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"]
Jul 28 19:31:09.833796 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:31:09.833753    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1794678e_72dd_47b2_ae07_d349a51f93a3.slice/crio-4de3c615de7076bd891372847048ceeb41d89608e4435970a28f8352e2c4e675 WatchSource:0}: Error finding container 4de3c615de7076bd891372847048ceeb41d89608e4435970a28f8352e2c4e675: Status 404 returned error can't find the container with id 4de3c615de7076bd891372847048ceeb41d89608e4435970a28f8352e2c4e675
Jul 28 19:31:09.835947 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.835904    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:31:09.836020 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.835982    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:31:09.836020 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.836014    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:31:09.856411 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:09.856379    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:10.306067 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:10.306027    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" event={"ID":"1794678e-72dd-47b2-ae07-d349a51f93a3","Type":"ContainerStarted","Data":"0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f"}
Jul 28 19:31:10.306234 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:10.306111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" event={"ID":"1794678e-72dd-47b2-ae07-d349a51f93a3","Type":"ContainerStarted","Data":"4de3c615de7076bd891372847048ceeb41d89608e4435970a28f8352e2c4e675"}
Jul 28 19:31:10.332874 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:10.332826    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" podStartSLOduration=1.332811537 podStartE2EDuration="1.332811537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:31:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:31:10.331220275 +0000 UTC m=+1711.792828002" watchObservedRunningTime="2026-07-28 19:31:10.332811537 +0000 UTC m=+1711.794419239"
Jul 28 19:31:10.687963 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:10.687899    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:10.692804 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:10.692776    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:11.310626 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:11.310600    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:11.311604 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:11.311587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:11.708036 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:11.707994    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:11.796332 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:11.796299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:11.830830 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:11.830797    2563 patch_prober.go:28] interesting pod/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 19:31:11.831007 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:11.830854    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 19:31:11.869900 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:11.869863    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:11.976421 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:11.976331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:12.354559 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:12.354531    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:31:12.360048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:12.360020    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:12.453831 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:12.453797    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:13.864851 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.864793    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:31:13.965204 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965172    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-envoy\") pod \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") "
Jul 28 19:31:13.965204 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965209    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-token\") pod \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") "
Jul 28 19:31:13.965423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965256    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-data\") pod \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") "
Jul 28 19:31:13.965423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965278    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-socket\") pod \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") "
Jul 28 19:31:13.965423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965298    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-podinfo\") pod \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") "
Jul 28 19:31:13.965423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965321    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istiod-ca-cert\") pod \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") "
Jul 28 19:31:13.965423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965373    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-certs\") pod \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") "
Jul 28 19:31:13.965423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965417    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9z24\" (UniqueName: \"kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-kube-api-access-n9z24\") pod \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") "
Jul 28 19:31:13.965711 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965444    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-credential-socket\") pod \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\" (UID: \"c17fedbd-2a41-42bb-91a0-76fa6410df0c\") "
Jul 28 19:31:13.965711 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965562    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c17fedbd-2a41-42bb-91a0-76fa6410df0c" (UID: "c17fedbd-2a41-42bb-91a0-76fa6410df0c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:31:13.965825 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965767    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:13.965880 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965810    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c17fedbd-2a41-42bb-91a0-76fa6410df0c" (UID: "c17fedbd-2a41-42bb-91a0-76fa6410df0c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:31:13.965964 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965896    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c17fedbd-2a41-42bb-91a0-76fa6410df0c" (UID: "c17fedbd-2a41-42bb-91a0-76fa6410df0c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:31:13.965964 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.965907    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c17fedbd-2a41-42bb-91a0-76fa6410df0c" (UID: "c17fedbd-2a41-42bb-91a0-76fa6410df0c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:31:13.966225 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.966192    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-data" (OuterVolumeSpecName: "istio-data") pod "c17fedbd-2a41-42bb-91a0-76fa6410df0c" (UID: "c17fedbd-2a41-42bb-91a0-76fa6410df0c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:31:13.967580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.967557    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c17fedbd-2a41-42bb-91a0-76fa6410df0c" (UID: "c17fedbd-2a41-42bb-91a0-76fa6410df0c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:31:13.967686 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.967665    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c17fedbd-2a41-42bb-91a0-76fa6410df0c" (UID: "c17fedbd-2a41-42bb-91a0-76fa6410df0c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:31:13.968443 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.968422    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-token" (OuterVolumeSpecName: "istio-token") pod "c17fedbd-2a41-42bb-91a0-76fa6410df0c" (UID: "c17fedbd-2a41-42bb-91a0-76fa6410df0c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:31:13.968443 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:13.968434    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-kube-api-access-n9z24" (OuterVolumeSpecName: "kube-api-access-n9z24") pod "c17fedbd-2a41-42bb-91a0-76fa6410df0c" (UID: "c17fedbd-2a41-42bb-91a0-76fa6410df0c"). InnerVolumeSpecName "kube-api-access-n9z24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:31:14.066372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.066338    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9z24\" (UniqueName: \"kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-kube-api-access-n9z24\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:14.066372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.066364    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:14.066372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.066374    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:14.066372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.066382    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:14.066627 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.066389    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:14.066627 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.066396    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:14.066627 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.066403    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c17fedbd-2a41-42bb-91a0-76fa6410df0c-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:14.066627 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.066410    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c17fedbd-2a41-42bb-91a0-76fa6410df0c-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:14.324775 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.324736    2563 generic.go:356] "Generic (PLEG): container finished" podID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerID="142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0" exitCode=0
Jul 28 19:31:14.324963 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.324780    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"
Jul 28 19:31:14.324963 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.324820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" event={"ID":"c17fedbd-2a41-42bb-91a0-76fa6410df0c","Type":"ContainerDied","Data":"142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0"}
Jul 28 19:31:14.324963 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.324875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp" event={"ID":"c17fedbd-2a41-42bb-91a0-76fa6410df0c","Type":"ContainerDied","Data":"d0809c6f9ba530e054af37dc97a3e7ffc613b00a239cdb823eb2b46ea648d37f"}
Jul 28 19:31:14.324963 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.324908    2563 scope.go:117] "RemoveContainer" containerID="142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0"
Jul 28 19:31:14.338460 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.334206    2563 scope.go:117] "RemoveContainer" containerID="dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f"
Jul 28 19:31:14.346988 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.346964    2563 scope.go:117] "RemoveContainer" containerID="142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0"
Jul 28 19:31:14.347370 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:31:14.347330    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0\": container with ID starting with 142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0 not found: ID does not exist" containerID="142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0"
Jul 28 19:31:14.347478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.347393    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0"} err="failed to get container status \"142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0\": rpc error: code = NotFound desc = could not find container \"142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0\": container with ID starting with 142bd8445795b6119b1b525a334a1346beaf43bc8c584e04dfe8e747feebe1b0 not found: ID does not exist"
Jul 28 19:31:14.347478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.347415    2563 scope.go:117] "RemoveContainer" containerID="dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f"
Jul 28 19:31:14.347702 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:31:14.347686    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f\": container with ID starting with dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f not found: ID does not exist" containerID="dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f"
Jul 28 19:31:14.347743 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.347708    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f"} err="failed to get container status \"dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f\": rpc error: code = NotFound desc = could not find container \"dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f\": container with ID starting with dbc32a30f79ff0280e0298c270d7b76d2e3ef803bb832e0e7a4e591c8c48195f not found: ID does not exist"
Jul 28 19:31:14.348557 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.348537    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"]
Jul 28 19:31:14.354408 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:14.354386    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-84845342-gw-openshift-default-7484f8d5dd-647fp"]
Jul 28 19:31:15.143127 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:15.143095    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" path="/var/lib/kubelet/pods/c17fedbd-2a41-42bb-91a0-76fa6410df0c/volumes"
Jul 28 19:31:22.050323 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:22.050283    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:31:26.312274 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:26.312188    2563 patch_prober.go:28] interesting pod/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 28 19:31:26.312274 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:26.312261    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 28 19:31:26.380286 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:26.380253    2563 generic.go:356] "Generic (PLEG): container finished" podID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerID="0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f" exitCode=0
Jul 28 19:31:26.380456 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:26.380294    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" event={"ID":"1794678e-72dd-47b2-ae07-d349a51f93a3","Type":"ContainerDied","Data":"0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f"}
Jul 28 19:31:26.380658 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:26.380645    2563 scope.go:117] "RemoveContainer" containerID="0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f"
Jul 28 19:31:26.381728 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:26.381701    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:31:26.381805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:26.381786    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:31:26.381848 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:26.381837    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:31:27.385901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:27.385865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" event={"ID":"1794678e-72dd-47b2-ae07-d349a51f93a3","Type":"ContainerStarted","Data":"8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f"}
Jul 28 19:31:27.687597 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:27.687490    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:27.687793 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:27.687766    2563 patch_prober.go:28] interesting pod/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 28 19:31:27.687840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:27.687821    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 28 19:31:28.688506 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:28.688468    2563 patch_prober.go:28] interesting pod/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 28 19:31:28.688954 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:28.688537    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 28 19:31:29.687977 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:29.687913    2563 patch_prober.go:28] interesting pod/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 28 19:31:29.688157 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:29.688002    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 28 19:31:30.687752 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:30.687716    2563 patch_prober.go:28] interesting pod/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 28 19:31:30.688138 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:30.687768    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 28 19:31:31.692604 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:31.692575    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:31.693009 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:31.692801    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:31.693606 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:31.693589    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:37.597934 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.597884    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"]
Jul 28 19:31:37.598340 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.598200    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy"
Jul 28 19:31:37.598340 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.598211    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy"
Jul 28 19:31:37.598340 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.598220    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy"
Jul 28 19:31:37.598340 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.598226    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy"
Jul 28 19:31:37.598340 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.598277    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy"
Jul 28 19:31:37.598340 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.598285    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17fedbd-2a41-42bb-91a0-76fa6410df0c" containerName="istio-proxy"
Jul 28 19:31:37.600185 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.600170    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"
Jul 28 19:31:37.603175 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.603153    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:31:37.603288 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.603155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:31:37.608994 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.608972    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"]
Jul 28 19:31:37.679685 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.679654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a362ea5-a02d-429c-b425-752fb60fe96a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-43235862-hk4xx\" (UID: \"0a362ea5-a02d-429c-b425-752fb60fe96a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"
Jul 28 19:31:37.780965 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.780899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a362ea5-a02d-429c-b425-752fb60fe96a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-43235862-hk4xx\" (UID: \"0a362ea5-a02d-429c-b425-752fb60fe96a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"
Jul 28 19:31:37.781536 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.781515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a362ea5-a02d-429c-b425-752fb60fe96a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-43235862-hk4xx\" (UID: \"0a362ea5-a02d-429c-b425-752fb60fe96a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"
Jul 28 19:31:37.912532 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:37.912444    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"
Jul 28 19:31:38.047514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:38.047488    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"]
Jul 28 19:31:38.049744 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:31:38.049719    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a362ea5_a02d_429c_b425_752fb60fe96a.slice/crio-1e116569737ebc671cbb5fa158124dcfffe179b04950287d05e6fa15996b71d9 WatchSource:0}: Error finding container 1e116569737ebc671cbb5fa158124dcfffe179b04950287d05e6fa15996b71d9: Status 404 returned error can't find the container with id 1e116569737ebc671cbb5fa158124dcfffe179b04950287d05e6fa15996b71d9
Jul 28 19:31:38.446062 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:38.445534    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx" event={"ID":"0a362ea5-a02d-429c-b425-752fb60fe96a","Type":"ContainerStarted","Data":"987dc2eb56f8b21a87d610de1a2865e99e585fcd20db8328384be2d1e0fa7e57"}
Jul 28 19:31:38.446062 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:38.445629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx" event={"ID":"0a362ea5-a02d-429c-b425-752fb60fe96a","Type":"ContainerStarted","Data":"1e116569737ebc671cbb5fa158124dcfffe179b04950287d05e6fa15996b71d9"}
Jul 28 19:31:38.460795 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:38.460690    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx" podStartSLOduration=1.460675369 podStartE2EDuration="1.460675369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:31:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:31:38.460093771 +0000 UTC m=+1739.921701476" watchObservedRunningTime="2026-07-28 19:31:38.460675369 +0000 UTC m=+1739.922283071"
Jul 28 19:31:39.451133 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:39.451044    2563 generic.go:356] "Generic (PLEG): container finished" podID="0a362ea5-a02d-429c-b425-752fb60fe96a" containerID="987dc2eb56f8b21a87d610de1a2865e99e585fcd20db8328384be2d1e0fa7e57" exitCode=0
Jul 28 19:31:39.451519 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:39.451130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx" event={"ID":"0a362ea5-a02d-429c-b425-752fb60fe96a","Type":"ContainerDied","Data":"987dc2eb56f8b21a87d610de1a2865e99e585fcd20db8328384be2d1e0fa7e57"}
Jul 28 19:31:40.581630 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:40.581606    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"
Jul 28 19:31:40.704862 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:40.704824    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a362ea5-a02d-429c-b425-752fb60fe96a-maas-api-service-ca\") pod \"0a362ea5-a02d-429c-b425-752fb60fe96a\" (UID: \"0a362ea5-a02d-429c-b425-752fb60fe96a\") "
Jul 28 19:31:40.705288 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:40.705241    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a362ea5-a02d-429c-b425-752fb60fe96a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a362ea5-a02d-429c-b425-752fb60fe96a" (UID: "0a362ea5-a02d-429c-b425-752fb60fe96a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:31:40.806153 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:40.806119    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a362ea5-a02d-429c-b425-752fb60fe96a-maas-api-service-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:41.460143 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:41.460049    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx" event={"ID":"0a362ea5-a02d-429c-b425-752fb60fe96a","Type":"ContainerDied","Data":"1e116569737ebc671cbb5fa158124dcfffe179b04950287d05e6fa15996b71d9"}
Jul 28 19:31:41.460143 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:41.460104    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e116569737ebc671cbb5fa158124dcfffe179b04950287d05e6fa15996b71d9"
Jul 28 19:31:41.460143 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:41.460063    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"
Jul 28 19:31:42.551237 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:42.551206    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:42.614655 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:42.614620    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:42.714721 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:42.714695    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:42.736256 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:42.736223    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"]
Jul 28 19:31:42.739053 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:42.739029    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-43235862-hk4xx"]
Jul 28 19:31:42.815355 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:42.815270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:43.143293 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:43.143260    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a362ea5-a02d-429c-b425-752fb60fe96a" path="/var/lib/kubelet/pods/0a362ea5-a02d-429c-b425-752fb60fe96a/volumes"
Jul 28 19:31:43.958534 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:43.958498    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"]
Jul 28 19:31:43.959049 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:43.958754    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy" containerID="cri-o://8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f" gracePeriod=30
Jul 28 19:31:43.985229 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:43.985194    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:44.193478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.193448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:44.422415 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.422376    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:44.715065 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.714987    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"]
Jul 28 19:31:44.715310 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.715295    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a362ea5-a02d-429c-b425-752fb60fe96a" containerName="revoke-keys"
Jul 28 19:31:44.715375 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.715312    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a362ea5-a02d-429c-b425-752fb60fe96a" containerName="revoke-keys"
Jul 28 19:31:44.715436 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.715380    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a362ea5-a02d-429c-b425-752fb60fe96a" containerName="revoke-keys"
Jul 28 19:31:44.717675 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.717656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.720637 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.720617    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-55d61f68-openshift-default-dockercfg-k2scp\""
Jul 28 19:31:44.731634 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.731611    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"]
Jul 28 19:31:44.753105 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.753066    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:44.839100 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.839062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-envoy\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.839100 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.839100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-podinfo\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.839318 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.839128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-credential-socket\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.839318 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.839145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istiod-ca-cert\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.839318 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.839212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgs57\" (UniqueName: \"kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-kube-api-access-sgs57\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.839318 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.839251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-data\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.839318 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.839288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-socket\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.839529 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.839330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-certs\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.839529 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.839367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-token\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.939933 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.939895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-envoy\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940095 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.939961    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-podinfo\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940095 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-credential-socket\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940095 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istiod-ca-cert\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940095 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgs57\" (UniqueName: \"kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-kube-api-access-sgs57\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940304 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-data\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940304 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-socket\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940452 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-certs\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940522 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940471    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-socket\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940522 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-token\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940522 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-credential-socket\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940678 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-data\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940799 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-certs\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.940856 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.940813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istiod-ca-cert\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.942368 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.942349    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-envoy\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.942601 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.942584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-podinfo\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.948854 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.948829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-token\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:44.949076 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:44.949055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgs57\" (UniqueName: \"kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-kube-api-access-sgs57\") pod \"e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") " pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:45.029066 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:45.028960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:45.038148 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:45.038112    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:45.172615 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:45.172590    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"]
Jul 28 19:31:45.174974 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:31:45.174947    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d34b536_4181_4b8a_bce9_f09cc79eb87f.slice/crio-9541f516c6d0830fcb7cfba08e200a4f9d91b41d466a462834b9272bf47e17ba WatchSource:0}: Error finding container 9541f516c6d0830fcb7cfba08e200a4f9d91b41d466a462834b9272bf47e17ba: Status 404 returned error can't find the container with id 9541f516c6d0830fcb7cfba08e200a4f9d91b41d466a462834b9272bf47e17ba
Jul 28 19:31:45.177477 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:45.177442    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:31:45.177548 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:45.177523    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:31:45.177598 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:45.177583    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:31:45.479751 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:45.479710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" event={"ID":"3d34b536-4181-4b8a-bce9-f09cc79eb87f","Type":"ContainerStarted","Data":"5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998"}
Jul 28 19:31:45.479961 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:45.479784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" event={"ID":"3d34b536-4181-4b8a-bce9-f09cc79eb87f","Type":"ContainerStarted","Data":"9541f516c6d0830fcb7cfba08e200a4f9d91b41d466a462834b9272bf47e17ba"}
Jul 28 19:31:45.506496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:45.506441    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" podStartSLOduration=1.506424554 podStartE2EDuration="1.506424554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:31:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:31:45.50531857 +0000 UTC m=+1746.966926274" watchObservedRunningTime="2026-07-28 19:31:45.506424554 +0000 UTC m=+1746.968032257"
Jul 28 19:31:46.029490 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:46.029446    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:46.034477 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:46.034449    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:46.341579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:46.341546    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:31:46.484580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:46.484553    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:46.485631 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:46.485615    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:31:46.695372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:46.695292    2563 patch_prober.go:28] interesting pod/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 19:31:46.695372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:46.695345    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 19:31:46.840558 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:46.840523    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:46.943531 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:46.943499    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:47.056797 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:47.056693    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:47.410705 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:47.410665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:47.567864 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:47.567836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:47.668222 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:47.668121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:31:49.211663 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.211640    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:49.384517 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.384485    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-credential-socket\") pod \"1794678e-72dd-47b2-ae07-d349a51f93a3\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") "
Jul 28 19:31:49.384703 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.384541    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-data\") pod \"1794678e-72dd-47b2-ae07-d349a51f93a3\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") "
Jul 28 19:31:49.384703 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.384561    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-podinfo\") pod \"1794678e-72dd-47b2-ae07-d349a51f93a3\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") "
Jul 28 19:31:49.384703 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.384594    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-token\") pod \"1794678e-72dd-47b2-ae07-d349a51f93a3\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") "
Jul 28 19:31:49.384703 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.384614    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7tzx\" (UniqueName: \"kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-kube-api-access-s7tzx\") pod \"1794678e-72dd-47b2-ae07-d349a51f93a3\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") "
Jul 28 19:31:49.384703 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.384634    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-socket\") pod \"1794678e-72dd-47b2-ae07-d349a51f93a3\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") "
Jul 28 19:31:49.384703 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.384672    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1794678e-72dd-47b2-ae07-d349a51f93a3-istiod-ca-cert\") pod \"1794678e-72dd-47b2-ae07-d349a51f93a3\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") "
Jul 28 19:31:49.385085 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.384725    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-envoy\") pod \"1794678e-72dd-47b2-ae07-d349a51f93a3\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") "
Jul 28 19:31:49.385085 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.384781    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-certs\") pod \"1794678e-72dd-47b2-ae07-d349a51f93a3\" (UID: \"1794678e-72dd-47b2-ae07-d349a51f93a3\") "
Jul 28 19:31:49.385085 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.385040    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1794678e-72dd-47b2-ae07-d349a51f93a3" (UID: "1794678e-72dd-47b2-ae07-d349a51f93a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:31:49.385253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.385202    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1794678e-72dd-47b2-ae07-d349a51f93a3" (UID: "1794678e-72dd-47b2-ae07-d349a51f93a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:31:49.385253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.385224    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1794678e-72dd-47b2-ae07-d349a51f93a3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1794678e-72dd-47b2-ae07-d349a51f93a3" (UID: "1794678e-72dd-47b2-ae07-d349a51f93a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:31:49.385408 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.385375    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-data" (OuterVolumeSpecName: "istio-data") pod "1794678e-72dd-47b2-ae07-d349a51f93a3" (UID: "1794678e-72dd-47b2-ae07-d349a51f93a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:31:49.385408 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.385396    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1794678e-72dd-47b2-ae07-d349a51f93a3" (UID: "1794678e-72dd-47b2-ae07-d349a51f93a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:31:49.387154 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.387117    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-token" (OuterVolumeSpecName: "istio-token") pod "1794678e-72dd-47b2-ae07-d349a51f93a3" (UID: "1794678e-72dd-47b2-ae07-d349a51f93a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:31:49.387154 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.387147    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1794678e-72dd-47b2-ae07-d349a51f93a3" (UID: "1794678e-72dd-47b2-ae07-d349a51f93a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:31:49.387423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.387397    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-kube-api-access-s7tzx" (OuterVolumeSpecName: "kube-api-access-s7tzx") pod "1794678e-72dd-47b2-ae07-d349a51f93a3" (UID: "1794678e-72dd-47b2-ae07-d349a51f93a3"). InnerVolumeSpecName "kube-api-access-s7tzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:31:49.387488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.387472    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1794678e-72dd-47b2-ae07-d349a51f93a3" (UID: "1794678e-72dd-47b2-ae07-d349a51f93a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:31:49.485598 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.485563    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:49.485598 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.485596    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7tzx\" (UniqueName: \"kubernetes.io/projected/1794678e-72dd-47b2-ae07-d349a51f93a3-kube-api-access-s7tzx\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:49.485598 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.485607    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:49.485821 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.485616    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1794678e-72dd-47b2-ae07-d349a51f93a3-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:49.485821 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.485624    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:49.485821 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.485631    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:49.485821 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.485638    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:49.485821 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.485646    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:49.485821 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.485653    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1794678e-72dd-47b2-ae07-d349a51f93a3-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:31:49.498730 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.498704    2563 generic.go:356] "Generic (PLEG): container finished" podID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerID="8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f" exitCode=0
Jul 28 19:31:49.498876 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.498743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" event={"ID":"1794678e-72dd-47b2-ae07-d349a51f93a3","Type":"ContainerDied","Data":"8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f"}
Jul 28 19:31:49.498876 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.498756    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"
Jul 28 19:31:49.498876 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.498787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9" event={"ID":"1794678e-72dd-47b2-ae07-d349a51f93a3","Type":"ContainerDied","Data":"4de3c615de7076bd891372847048ceeb41d89608e4435970a28f8352e2c4e675"}
Jul 28 19:31:49.498876 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.498810    2563 scope.go:117] "RemoveContainer" containerID="8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f"
Jul 28 19:31:49.507737 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.507718    2563 scope.go:117] "RemoveContainer" containerID="0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f"
Jul 28 19:31:49.515449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.515423    2563 scope.go:117] "RemoveContainer" containerID="8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f"
Jul 28 19:31:49.515698 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:31:49.515678    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f\": container with ID starting with 8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f not found: ID does not exist" containerID="8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f"
Jul 28 19:31:49.515763 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.515707    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f"} err="failed to get container status \"8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f\": rpc error: code = NotFound desc = could not find container \"8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f\": container with ID starting with 8df830022ef043c2fc99a9b10894f6b01fddd79fbf13e92288c0a18436daa51f not found: ID does not exist"
Jul 28 19:31:49.515763 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.515725    2563 scope.go:117] "RemoveContainer" containerID="0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f"
Jul 28 19:31:49.515979 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:31:49.515956    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f\": container with ID starting with 0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f not found: ID does not exist" containerID="0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f"
Jul 28 19:31:49.516021 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.515990    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f"} err="failed to get container status \"0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f\": rpc error: code = NotFound desc = could not find container \"0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f\": container with ID starting with 0ba45d9c03f68bee714248843262de0a00a839141e5fa1b62a81e4694ee2ef7f not found: ID does not exist"
Jul 28 19:31:49.539040 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.539005    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"]
Jul 28 19:31:49.550702 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:49.550678    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-43235862-openshift-default-84c7dd5878-w82m9"]
Jul 28 19:31:51.148344 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:51.146195    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" path="/var/lib/kubelet/pods/1794678e-72dd-47b2-ae07-d349a51f93a3/volumes"
Jul 28 19:31:56.045418 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:31:56.045383    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:32:00.550814 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:00.550782    2563 generic.go:356] "Generic (PLEG): container finished" podID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerID="5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998" exitCode=0
Jul 28 19:32:00.551348 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:00.550859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" event={"ID":"3d34b536-4181-4b8a-bce9-f09cc79eb87f","Type":"ContainerDied","Data":"5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998"}
Jul 28 19:32:00.551446 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:00.551427    2563 scope.go:117] "RemoveContainer" containerID="5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998"
Jul 28 19:32:00.552572 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:00.552544    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:32:00.552638 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:00.552618    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:32:00.552687 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:00.552659    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:32:01.485128 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:01.485099    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:32:01.556993 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:01.556956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" event={"ID":"3d34b536-4181-4b8a-bce9-f09cc79eb87f","Type":"ContainerStarted","Data":"cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2"}
Jul 28 19:32:02.029614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:02.029574    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:32:02.030817 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:02.030790    2563 patch_prober.go:28] interesting pod/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 28 19:32:02.030938 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:02.030841    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 28 19:32:03.029674 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:03.029637    2563 patch_prober.go:28] interesting pod/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 28 19:32:03.030110 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:03.029709    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 28 19:32:04.030002 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:04.029968    2563 patch_prober.go:28] interesting pod/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 28 19:32:04.030393 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:04.030019    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 28 19:32:05.029649 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:05.029611    2563 patch_prober.go:28] interesting pod/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 28 19:32:05.029835 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:05.029666    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 28 19:32:06.033867 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:06.033834    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:32:06.034313 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:06.034236    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:32:06.034881 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:06.034863    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:32:09.973835 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:09.973793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:10.033721 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:10.033689    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:16.887777 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.887739    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"]
Jul 28 19:32:16.888328 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.888200    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy"
Jul 28 19:32:16.888328 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.888219    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy"
Jul 28 19:32:16.888328 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.888248    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy"
Jul 28 19:32:16.888328 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.888258    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy"
Jul 28 19:32:16.888541 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.888364    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy"
Jul 28 19:32:16.888541 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.888382    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1794678e-72dd-47b2-ae07-d349a51f93a3" containerName="istio-proxy"
Jul 28 19:32:16.890279 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.890254    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"
Jul 28 19:32:16.893088 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.893069    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:32:16.893204 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.893069    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:32:16.898805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:16.898779    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"]
Jul 28 19:32:17.006614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:17.006576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d2d81d-9665-4b90-8680-ed654fa0aa64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-55d61f68-sqb62\" (UID: \"18d2d81d-9665-4b90-8680-ed654fa0aa64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"
Jul 28 19:32:17.107250 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:17.107212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d2d81d-9665-4b90-8680-ed654fa0aa64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-55d61f68-sqb62\" (UID: \"18d2d81d-9665-4b90-8680-ed654fa0aa64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"
Jul 28 19:32:17.107862 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:17.107836    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d2d81d-9665-4b90-8680-ed654fa0aa64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-55d61f68-sqb62\" (UID: \"18d2d81d-9665-4b90-8680-ed654fa0aa64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"
Jul 28 19:32:17.202606 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:17.202520    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"
Jul 28 19:32:17.333388 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:17.333362    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"]
Jul 28 19:32:17.335880 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:32:17.335848    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d2d81d_9665_4b90_8680_ed654fa0aa64.slice/crio-7aa3c4f3e2590b6e4721917e5ed2dbe6b712e23b61b986e0a46765980e48933a WatchSource:0}: Error finding container 7aa3c4f3e2590b6e4721917e5ed2dbe6b712e23b61b986e0a46765980e48933a: Status 404 returned error can't find the container with id 7aa3c4f3e2590b6e4721917e5ed2dbe6b712e23b61b986e0a46765980e48933a
Jul 28 19:32:17.628692 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:17.628654    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62" event={"ID":"18d2d81d-9665-4b90-8680-ed654fa0aa64","Type":"ContainerStarted","Data":"7328d13ff7491c15171e040b5c91fef68f74ddab5fc1cb726e72f289e8ab8aba"}
Jul 28 19:32:17.628811 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:17.628715    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62" event={"ID":"18d2d81d-9665-4b90-8680-ed654fa0aa64","Type":"ContainerStarted","Data":"7aa3c4f3e2590b6e4721917e5ed2dbe6b712e23b61b986e0a46765980e48933a"}
Jul 28 19:32:17.644517 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:17.644474    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62" podStartSLOduration=1.644458687 podStartE2EDuration="1.644458687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:32:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:32:17.643125616 +0000 UTC m=+1779.104733319" watchObservedRunningTime="2026-07-28 19:32:17.644458687 +0000 UTC m=+1779.106066389"
Jul 28 19:32:18.634997 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:18.634964    2563 generic.go:356] "Generic (PLEG): container finished" podID="18d2d81d-9665-4b90-8680-ed654fa0aa64" containerID="7328d13ff7491c15171e040b5c91fef68f74ddab5fc1cb726e72f289e8ab8aba" exitCode=0
Jul 28 19:32:18.635396 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:18.635047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62" event={"ID":"18d2d81d-9665-4b90-8680-ed654fa0aa64","Type":"ContainerDied","Data":"7328d13ff7491c15171e040b5c91fef68f74ddab5fc1cb726e72f289e8ab8aba"}
Jul 28 19:32:19.767502 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:19.767479    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"
Jul 28 19:32:19.830859 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:19.830823    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d2d81d-9665-4b90-8680-ed654fa0aa64-maas-api-service-ca\") pod \"18d2d81d-9665-4b90-8680-ed654fa0aa64\" (UID: \"18d2d81d-9665-4b90-8680-ed654fa0aa64\") "
Jul 28 19:32:19.831223 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:19.831199    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18d2d81d-9665-4b90-8680-ed654fa0aa64-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "18d2d81d-9665-4b90-8680-ed654fa0aa64" (UID: "18d2d81d-9665-4b90-8680-ed654fa0aa64"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:32:19.931864 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:19.931778    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d2d81d-9665-4b90-8680-ed654fa0aa64-maas-api-service-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:32:20.645571 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:20.645544    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"
Jul 28 19:32:20.645571 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:20.645550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62" event={"ID":"18d2d81d-9665-4b90-8680-ed654fa0aa64","Type":"ContainerDied","Data":"7aa3c4f3e2590b6e4721917e5ed2dbe6b712e23b61b986e0a46765980e48933a"}
Jul 28 19:32:20.645764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:20.645591    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7aa3c4f3e2590b6e4721917e5ed2dbe6b712e23b61b986e0a46765980e48933a"
Jul 28 19:32:27.597587 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:27.597552    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:27.696450 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:27.696414    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:27.782375 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:27.782341    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"]
Jul 28 19:32:27.785526 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:27.785492    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55d61f68-sqb62"]
Jul 28 19:32:27.785692 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:27.785669    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:27.859305 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:27.859207    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:27.918554 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:27.918520    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"]
Jul 28 19:32:27.918815 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:27.918788    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy" containerID="cri-o://cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2" gracePeriod=30
Jul 28 19:32:27.950320 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:27.950289    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:28.174399 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:28.174319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:28.678888 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:28.678856    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:28.822699 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:28.822665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:28.979511 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:28.979431    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:29.143322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:29.143254    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18d2d81d-9665-4b90-8680-ed654fa0aa64" path="/var/lib/kubelet/pods/18d2d81d-9665-4b90-8680-ed654fa0aa64/volumes"
Jul 28 19:32:30.281970 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:30.281897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:31.445902 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:31.445869    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:32:33.162263 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.162244    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:32:33.246710 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.246621    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgs57\" (UniqueName: \"kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-kube-api-access-sgs57\") pod \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") "
Jul 28 19:32:33.246710 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.246690    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-data\") pod \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") "
Jul 28 19:32:33.246966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.246722    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-credential-socket\") pod \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") "
Jul 28 19:32:33.246966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.246755    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-envoy\") pod \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") "
Jul 28 19:32:33.246966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.246782    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-socket\") pod \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") "
Jul 28 19:32:33.246966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.246804    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-token\") pod \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") "
Jul 28 19:32:33.246966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.246838    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-podinfo\") pod \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") "
Jul 28 19:32:33.246966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.246866    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-certs\") pod \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") "
Jul 28 19:32:33.246966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.246911    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istiod-ca-cert\") pod \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\" (UID: \"3d34b536-4181-4b8a-bce9-f09cc79eb87f\") "
Jul 28 19:32:33.247319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.247066    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d34b536-4181-4b8a-bce9-f09cc79eb87f" (UID: "3d34b536-4181-4b8a-bce9-f09cc79eb87f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:32:33.247319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.247098    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d34b536-4181-4b8a-bce9-f09cc79eb87f" (UID: "3d34b536-4181-4b8a-bce9-f09cc79eb87f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:32:33.247432 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.247337    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d34b536-4181-4b8a-bce9-f09cc79eb87f" (UID: "3d34b536-4181-4b8a-bce9-f09cc79eb87f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:32:33.247518 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.247492    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d34b536-4181-4b8a-bce9-f09cc79eb87f" (UID: "3d34b536-4181-4b8a-bce9-f09cc79eb87f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:32:33.247703 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.247680    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:32:33.247837 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.247706    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:32:33.247837 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.247720    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:32:33.247837 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.247732    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:32:33.248011 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.247889    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d34b536-4181-4b8a-bce9-f09cc79eb87f" (UID: "3d34b536-4181-4b8a-bce9-f09cc79eb87f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:32:33.249132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.249106    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-kube-api-access-sgs57" (OuterVolumeSpecName: "kube-api-access-sgs57") pod "3d34b536-4181-4b8a-bce9-f09cc79eb87f" (UID: "3d34b536-4181-4b8a-bce9-f09cc79eb87f"). InnerVolumeSpecName "kube-api-access-sgs57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:32:33.249387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.249370    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d34b536-4181-4b8a-bce9-f09cc79eb87f" (UID: "3d34b536-4181-4b8a-bce9-f09cc79eb87f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:32:33.249652 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.249630    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d34b536-4181-4b8a-bce9-f09cc79eb87f" (UID: "3d34b536-4181-4b8a-bce9-f09cc79eb87f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:32:33.249693 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.249670    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d34b536-4181-4b8a-bce9-f09cc79eb87f" (UID: "3d34b536-4181-4b8a-bce9-f09cc79eb87f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:32:33.348283 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.348248    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgs57\" (UniqueName: \"kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-kube-api-access-sgs57\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:32:33.348283 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.348277    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:32:33.348283 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.348287    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:32:33.348283 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.348294    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:32:33.348543 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.348303    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d34b536-4181-4b8a-bce9-f09cc79eb87f-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:32:33.704197 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.704163    2563 generic.go:356] "Generic (PLEG): container finished" podID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerID="cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2" exitCode=0
Jul 28 19:32:33.704380 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.704211    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"
Jul 28 19:32:33.704380 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.704219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" event={"ID":"3d34b536-4181-4b8a-bce9-f09cc79eb87f","Type":"ContainerDied","Data":"cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2"}
Jul 28 19:32:33.704380 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.704263    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj" event={"ID":"3d34b536-4181-4b8a-bce9-f09cc79eb87f","Type":"ContainerDied","Data":"9541f516c6d0830fcb7cfba08e200a4f9d91b41d466a462834b9272bf47e17ba"}
Jul 28 19:32:33.704380 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.704287    2563 scope.go:117] "RemoveContainer" containerID="cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2"
Jul 28 19:32:33.713007 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.712990    2563 scope.go:117] "RemoveContainer" containerID="5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998"
Jul 28 19:32:33.720913 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.720890    2563 scope.go:117] "RemoveContainer" containerID="cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2"
Jul 28 19:32:33.721214 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:32:33.721195    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2\": container with ID starting with cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2 not found: ID does not exist" containerID="cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2"
Jul 28 19:32:33.721304 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.721225    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2"} err="failed to get container status \"cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2\": rpc error: code = NotFound desc = could not find container \"cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2\": container with ID starting with cae949e586504e72d404196365745a52a5fe7dd354c9a67a854ae0de3a3ea6a2 not found: ID does not exist"
Jul 28 19:32:33.721304 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.721248    2563 scope.go:117] "RemoveContainer" containerID="5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998"
Jul 28 19:32:33.721518 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:32:33.721500    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998\": container with ID starting with 5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998 not found: ID does not exist" containerID="5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998"
Jul 28 19:32:33.721560 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.721525    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998"} err="failed to get container status \"5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998\": rpc error: code = NotFound desc = could not find container \"5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998\": container with ID starting with 5d737ed3743369e60aef08640c7b049e859c317731c122f8ef86886e79dc6998 not found: ID does not exist"
Jul 28 19:32:33.741348 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.741318    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"]
Jul 28 19:32:33.745614 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:33.745586    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-55d61f68-openshift-default-64596d8876-fh5xj"]
Jul 28 19:32:35.144087 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:35.144057    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" path="/var/lib/kubelet/pods/3d34b536-4181-4b8a-bce9-f09cc79eb87f/volumes"
Jul 28 19:32:36.166962 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:36.166904    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:36.271253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:36.271206    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:36.366723 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:36.366693    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:36.467355 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:36.467271    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:36.863367 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:36.863339    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:32:39.059257 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:39.059228    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:32:39.063362 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:39.063344    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:32:42.042753 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:32:42.042718    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:33:02.790289 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:02.790244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:09.555222 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:09.555179    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:09.663162 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:09.663124    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:09.754113 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:09.754074    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:25.638605 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:25.638567    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:31.274425 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:31.274387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:31.331296 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:31.331258    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:31.403053 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:31.403017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:31.502555 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:31.502521    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:41.973960 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:41.973904    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:42.772402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:42.772363    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:49.251123 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:49.251084    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:49.355867 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:49.355829    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:49.451145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:49.451111    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:49.549411 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:49.549312    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:52.742868 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:52.742828    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:33:54.669824 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:54.669784    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:54.770757 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:54.770717    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:54.868332 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:54.868273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:33:56.589060 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:33:56.589022    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:02.153445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:02.153366    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:34:02.707583 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:02.707539    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:02.804727 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:02.804687    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:03.420798 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:03.420761    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:05.194904 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:05.194869    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:09.138590 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.138555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:09.238715 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.238677    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:09.365962 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.365885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:09.639883 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.639856    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"]
Jul 28 19:34:09.640216 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.640202    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy"
Jul 28 19:34:09.640271 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.640218    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy"
Jul 28 19:34:09.640271 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.640235    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy"
Jul 28 19:34:09.640271 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.640241    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy"
Jul 28 19:34:09.640271 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.640251    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d2d81d-9665-4b90-8680-ed654fa0aa64" containerName="revoke-keys"
Jul 28 19:34:09.640271 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.640256    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d2d81d-9665-4b90-8680-ed654fa0aa64" containerName="revoke-keys"
Jul 28 19:34:09.640441 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.640317    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy"
Jul 28 19:34:09.640441 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.640325    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d2d81d-9665-4b90-8680-ed654fa0aa64" containerName="revoke-keys"
Jul 28 19:34:09.640441 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.640333    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d34b536-4181-4b8a-bce9-f09cc79eb87f" containerName="istio-proxy"
Jul 28 19:34:09.642145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.642129    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.645880 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.645860    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e72a09a2-openshift-default-dockercfg-6bc52\""
Jul 28 19:34:09.671300 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.671265    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"]
Jul 28 19:34:09.675192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.675166    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:09.741221 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.741182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-data\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.741221 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.741223    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05cee881-d937-45ed-99c9-17337af8d70b-istio-podinfo\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.741423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.741251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwrck\" (UniqueName: \"kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-kube-api-access-dwrck\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.741423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.741277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-socket\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.741423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.741330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-envoy\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.741423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.741366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-credential-socket\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.741423 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.741391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-istio-token\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.741576 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.741425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-certs\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.741576 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.741498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05cee881-d937-45ed-99c9-17337af8d70b-istiod-ca-cert\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.842634 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.842592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05cee881-d937-45ed-99c9-17337af8d70b-istiod-ca-cert\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.842849 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.842657    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-data\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.842849 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.842685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05cee881-d937-45ed-99c9-17337af8d70b-istio-podinfo\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.842849 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.842712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwrck\" (UniqueName: \"kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-kube-api-access-dwrck\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.842849 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.842736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-socket\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.842849 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.842762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-envoy\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.843112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.842944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-credential-socket\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.843112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.842999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-istio-token\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.843112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.843055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-certs\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.843265 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.843220    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-data\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.843339 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.843285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-credential-socket\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.843339 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.843327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-socket\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.843456 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.843398    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-certs\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.843512 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.843456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05cee881-d937-45ed-99c9-17337af8d70b-istiod-ca-cert\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.845444 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.845423    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05cee881-d937-45ed-99c9-17337af8d70b-istio-podinfo\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.845444 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.845443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-envoy\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.854382 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.854361    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwrck\" (UniqueName: \"kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-kube-api-access-dwrck\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.854629 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.854611    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-istio-token\") pod \"e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") " pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:09.953228 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:09.953116    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:10.085685 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:10.085659    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"]
Jul 28 19:34:10.087665 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:34:10.087637    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05cee881_d937_45ed_99c9_17337af8d70b.slice/crio-b301e27bf940a4a79e687138cf9819d9881a0926dabfe4b29ef14e9a44b1836a WatchSource:0}: Error finding container b301e27bf940a4a79e687138cf9819d9881a0926dabfe4b29ef14e9a44b1836a: Status 404 returned error can't find the container with id b301e27bf940a4a79e687138cf9819d9881a0926dabfe4b29ef14e9a44b1836a
Jul 28 19:34:10.089739 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:10.089699    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:34:10.089820 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:10.089777    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:34:10.089820 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:10.089803    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:34:10.121879 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:10.121855    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" event={"ID":"05cee881-d937-45ed-99c9-17337af8d70b","Type":"ContainerStarted","Data":"b301e27bf940a4a79e687138cf9819d9881a0926dabfe4b29ef14e9a44b1836a"}
Jul 28 19:34:10.226725 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:10.226633    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:11.127443 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:11.127408    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" event={"ID":"05cee881-d937-45ed-99c9-17337af8d70b","Type":"ContainerStarted","Data":"84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833"}
Jul 28 19:34:11.147526 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:11.147474    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" podStartSLOduration=2.1474584820000002 podStartE2EDuration="2.147458482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:11.146571788 +0000 UTC m=+1892.608179492" watchObservedRunningTime="2026-07-28 19:34:11.147458482 +0000 UTC m=+1892.609066187"
Jul 28 19:34:11.953875 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:11.953836    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:11.958457 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:11.958435    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:12.132080 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:12.132048    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:12.133109 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:12.133086    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:12.142351 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:12.142324    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:34:13.090012 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:13.089980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:13.191871 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:13.191794    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:13.289316 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:13.289270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:13.610197 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:13.610159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:13.783059 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:13.783026    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:13.883242 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:13.883156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:23.440203 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:23.440167    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:34:27.133479 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:27.133440    2563 patch_prober.go:28] interesting pod/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 28 19:34:27.133878 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:27.133517    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 28 19:34:27.194794 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:27.194758    2563 generic.go:356] "Generic (PLEG): container finished" podID="05cee881-d937-45ed-99c9-17337af8d70b" containerID="84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833" exitCode=0
Jul 28 19:34:27.194984 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:27.194832    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" event={"ID":"05cee881-d937-45ed-99c9-17337af8d70b","Type":"ContainerDied","Data":"84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833"}
Jul 28 19:34:27.195220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:27.195207    2563 scope.go:117] "RemoveContainer" containerID="84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833"
Jul 28 19:34:27.196233 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:27.196202    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:34:27.196347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:27.196287    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:34:27.196347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:27.196334    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:34:28.216952 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:28.216564    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" event={"ID":"05cee881-d937-45ed-99c9-17337af8d70b","Type":"ContainerStarted","Data":"5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2"}
Jul 28 19:34:28.954143 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:28.954103    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:28.954392 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:28.954369    2563 patch_prober.go:28] interesting pod/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 28 19:34:28.954447 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:28.954416    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 28 19:34:29.953648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:29.953607    2563 patch_prober.go:28] interesting pod/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 28 19:34:29.954059 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:29.953663    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 28 19:34:30.953692 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:30.953650    2563 patch_prober.go:28] interesting pod/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 28 19:34:30.954115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:30.953707    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 28 19:34:31.953678 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:31.953644    2563 patch_prober.go:28] interesting pod/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 28 19:34:31.953856 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:31.953710    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 28 19:34:32.957801 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:32.957769    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:32.958183 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:32.958129    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:32.958825 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:32.958807    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:34:38.064037 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:38.063999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:38.713343 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:38.713303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:38.814801 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:38.814766    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:39.127276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:39.127225    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:44.662776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:44.662732    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:44.763081 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:44.763051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:45.012152 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:45.012072    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"]
Jul 28 19:34:45.014427 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:45.014412    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"
Jul 28 19:34:45.017130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:45.017113    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:34:45.017211 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:45.017176    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:34:45.022913 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:45.022885    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"]
Jul 28 19:34:45.152639 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:45.152608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ece8b22-3f49-483f-b5ec-54eaae970199-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2\" (UID: \"6ece8b22-3f49-483f-b5ec-54eaae970199\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"
Jul 28 19:34:45.253050 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:45.253013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ece8b22-3f49-483f-b5ec-54eaae970199-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2\" (UID: \"6ece8b22-3f49-483f-b5ec-54eaae970199\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"
Jul 28 19:34:45.253624 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:45.253598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ece8b22-3f49-483f-b5ec-54eaae970199-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2\" (UID: \"6ece8b22-3f49-483f-b5ec-54eaae970199\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"
Jul 28 19:34:45.326487 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:45.326463    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"
Jul 28 19:34:45.454117 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:45.454079    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"]
Jul 28 19:34:45.456094 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:34:45.456067    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ece8b22_3f49_483f_b5ec_54eaae970199.slice/crio-e7f56b168eb4538c99ab5e7adcac831b788155034a1fbe9f28227ffabb581873 WatchSource:0}: Error finding container e7f56b168eb4538c99ab5e7adcac831b788155034a1fbe9f28227ffabb581873: Status 404 returned error can't find the container with id e7f56b168eb4538c99ab5e7adcac831b788155034a1fbe9f28227ffabb581873
Jul 28 19:34:46.291385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:46.291282    2563 generic.go:356] "Generic (PLEG): container finished" podID="6ece8b22-3f49-483f-b5ec-54eaae970199" containerID="45cf0d83ac258bb9ca5823a4af7009c609c0ba08da1636dcdb3c0e2c988e4cf0" exitCode=0
Jul 28 19:34:46.291385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:46.291368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2" event={"ID":"6ece8b22-3f49-483f-b5ec-54eaae970199","Type":"ContainerDied","Data":"45cf0d83ac258bb9ca5823a4af7009c609c0ba08da1636dcdb3c0e2c988e4cf0"}
Jul 28 19:34:46.291789 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:46.291422    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2" event={"ID":"6ece8b22-3f49-483f-b5ec-54eaae970199","Type":"ContainerStarted","Data":"e7f56b168eb4538c99ab5e7adcac831b788155034a1fbe9f28227ffabb581873"}
Jul 28 19:34:47.432772 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:47.432749    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"
Jul 28 19:34:47.573651 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:47.573612    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ece8b22-3f49-483f-b5ec-54eaae970199-maas-api-service-ca\") pod \"6ece8b22-3f49-483f-b5ec-54eaae970199\" (UID: \"6ece8b22-3f49-483f-b5ec-54eaae970199\") "
Jul 28 19:34:47.573984 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:47.573962    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ece8b22-3f49-483f-b5ec-54eaae970199-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ece8b22-3f49-483f-b5ec-54eaae970199" (UID: "6ece8b22-3f49-483f-b5ec-54eaae970199"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:34:47.674879 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:47.674851    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ece8b22-3f49-483f-b5ec-54eaae970199-maas-api-service-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:34:48.301814 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:48.301785    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"
Jul 28 19:34:48.301814 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:48.301810    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2" event={"ID":"6ece8b22-3f49-483f-b5ec-54eaae970199","Type":"ContainerDied","Data":"e7f56b168eb4538c99ab5e7adcac831b788155034a1fbe9f28227ffabb581873"}
Jul 28 19:34:48.302047 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:48.301853    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7f56b168eb4538c99ab5e7adcac831b788155034a1fbe9f28227ffabb581873"
Jul 28 19:34:55.458169 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:55.458138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:55.556822 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:55.556771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:55.618988 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:55.618950    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"]
Jul 28 19:34:55.621881 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:55.621849    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e72a09a2-bhrt2"]
Jul 28 19:34:55.622033 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:55.621958    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:55.678350 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:55.678312    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:56.049262 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:56.049228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:34:57.143313 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:57.143278    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ece8b22-3f49-483f-b5ec-54eaae970199" path="/var/lib/kubelet/pods/6ece8b22-3f49-483f-b5ec-54eaae970199/volumes"
Jul 28 19:34:58.544845 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:34:58.544812    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:35:01.665977 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:01.665947    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:01.767034 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:01.766999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:01.896695 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:01.896662    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"]
Jul 28 19:35:01.896944 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:01.896903    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy" containerID="cri-o://5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2" gracePeriod=30
Jul 28 19:35:01.919259 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:01.919171    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:02.181632 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:02.181549    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:02.286617 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:02.286583    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:02.382869 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:02.382833    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:02.496421 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:02.496335    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:02.960232 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:02.960196    2563 patch_prober.go:28] interesting pod/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 19:35:02.960598 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:02.960263    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 19:35:07.139993 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.139970    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:35:07.233385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233290    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-envoy\") pod \"05cee881-d937-45ed-99c9-17337af8d70b\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") "
Jul 28 19:35:07.233385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233327    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-socket\") pod \"05cee881-d937-45ed-99c9-17337af8d70b\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") "
Jul 28 19:35:07.233385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233350    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-istio-token\") pod \"05cee881-d937-45ed-99c9-17337af8d70b\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") "
Jul 28 19:35:07.233385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233379    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05cee881-d937-45ed-99c9-17337af8d70b-istiod-ca-cert\") pod \"05cee881-d937-45ed-99c9-17337af8d70b\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") "
Jul 28 19:35:07.233719 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233404    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-certs\") pod \"05cee881-d937-45ed-99c9-17337af8d70b\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") "
Jul 28 19:35:07.233719 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233472    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-data\") pod \"05cee881-d937-45ed-99c9-17337af8d70b\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") "
Jul 28 19:35:07.233719 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233501    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwrck\" (UniqueName: \"kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-kube-api-access-dwrck\") pod \"05cee881-d937-45ed-99c9-17337af8d70b\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") "
Jul 28 19:35:07.233719 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233534    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05cee881-d937-45ed-99c9-17337af8d70b-istio-podinfo\") pod \"05cee881-d937-45ed-99c9-17337af8d70b\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") "
Jul 28 19:35:07.233719 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233557    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-credential-socket\") pod \"05cee881-d937-45ed-99c9-17337af8d70b\" (UID: \"05cee881-d937-45ed-99c9-17337af8d70b\") "
Jul 28 19:35:07.233719 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233624    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "05cee881-d937-45ed-99c9-17337af8d70b" (UID: "05cee881-d937-45ed-99c9-17337af8d70b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:35:07.234068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233813    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "05cee881-d937-45ed-99c9-17337af8d70b" (UID: "05cee881-d937-45ed-99c9-17337af8d70b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:35:07.234068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233853    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:35:07.234068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233876    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "05cee881-d937-45ed-99c9-17337af8d70b" (UID: "05cee881-d937-45ed-99c9-17337af8d70b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:35:07.234068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.233947    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05cee881-d937-45ed-99c9-17337af8d70b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "05cee881-d937-45ed-99c9-17337af8d70b" (UID: "05cee881-d937-45ed-99c9-17337af8d70b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:35:07.234740 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.234715    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-data" (OuterVolumeSpecName: "istio-data") pod "05cee881-d937-45ed-99c9-17337af8d70b" (UID: "05cee881-d937-45ed-99c9-17337af8d70b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:35:07.235653 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.235634    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "05cee881-d937-45ed-99c9-17337af8d70b" (UID: "05cee881-d937-45ed-99c9-17337af8d70b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:35:07.235728 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.235710    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-istio-token" (OuterVolumeSpecName: "istio-token") pod "05cee881-d937-45ed-99c9-17337af8d70b" (UID: "05cee881-d937-45ed-99c9-17337af8d70b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:35:07.236295 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.236271    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05cee881-d937-45ed-99c9-17337af8d70b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "05cee881-d937-45ed-99c9-17337af8d70b" (UID: "05cee881-d937-45ed-99c9-17337af8d70b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:35:07.236295 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.236274    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-kube-api-access-dwrck" (OuterVolumeSpecName: "kube-api-access-dwrck") pod "05cee881-d937-45ed-99c9-17337af8d70b" (UID: "05cee881-d937-45ed-99c9-17337af8d70b"). InnerVolumeSpecName "kube-api-access-dwrck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:35:07.335018 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.334986    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:35:07.335018 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.335014    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:35:07.335018 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.335022    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05cee881-d937-45ed-99c9-17337af8d70b-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:35:07.335247 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.335032    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:35:07.335247 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.335041    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:35:07.335247 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.335049    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwrck\" (UniqueName: \"kubernetes.io/projected/05cee881-d937-45ed-99c9-17337af8d70b-kube-api-access-dwrck\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:35:07.335247 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.335059    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05cee881-d937-45ed-99c9-17337af8d70b-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:35:07.335247 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.335066    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05cee881-d937-45ed-99c9-17337af8d70b-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:35:07.398439 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.398402    2563 generic.go:356] "Generic (PLEG): container finished" podID="05cee881-d937-45ed-99c9-17337af8d70b" containerID="5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2" exitCode=0
Jul 28 19:35:07.398603 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.398459    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"
Jul 28 19:35:07.398603 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.398475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" event={"ID":"05cee881-d937-45ed-99c9-17337af8d70b","Type":"ContainerDied","Data":"5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2"}
Jul 28 19:35:07.398603 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.398545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk" event={"ID":"05cee881-d937-45ed-99c9-17337af8d70b","Type":"ContainerDied","Data":"b301e27bf940a4a79e687138cf9819d9881a0926dabfe4b29ef14e9a44b1836a"}
Jul 28 19:35:07.398603 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.398567    2563 scope.go:117] "RemoveContainer" containerID="5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2"
Jul 28 19:35:07.407489 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.407470    2563 scope.go:117] "RemoveContainer" containerID="84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833"
Jul 28 19:35:07.415688 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.415660    2563 scope.go:117] "RemoveContainer" containerID="5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2"
Jul 28 19:35:07.415978 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:35:07.415956    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2\": container with ID starting with 5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2 not found: ID does not exist" containerID="5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2"
Jul 28 19:35:07.416043 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.415987    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2"} err="failed to get container status \"5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2\": rpc error: code = NotFound desc = could not find container \"5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2\": container with ID starting with 5c0e757bfc63ccaa90f3acf91a056ecf797156cb02b1d0592cd3f38234529bd2 not found: ID does not exist"
Jul 28 19:35:07.416043 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.416005    2563 scope.go:117] "RemoveContainer" containerID="84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833"
Jul 28 19:35:07.416288 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:35:07.416272    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833\": container with ID starting with 84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833 not found: ID does not exist" containerID="84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833"
Jul 28 19:35:07.416345 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.416293    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833"} err="failed to get container status \"84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833\": rpc error: code = NotFound desc = could not find container \"84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833\": container with ID starting with 84c078f94a20660f25a0b1d45cbeafa11b401693a06eccac7daafa8ad8aef833 not found: ID does not exist"
Jul 28 19:35:07.420140 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.420116    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"]
Jul 28 19:35:07.425508 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:07.425487    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e72a09a2-openshift-default-56549555d4-2lqnk"]
Jul 28 19:35:08.096275 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:08.096243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:08.154795 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:08.154761    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:08.236277 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:08.236247    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:08.334416 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:08.334383    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:08.447188 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:08.447101    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:09.142764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:09.142731    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05cee881-d937-45ed-99c9-17337af8d70b" path="/var/lib/kubelet/pods/05cee881-d937-45ed-99c9-17337af8d70b/volumes"
Jul 28 19:35:14.140959 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:14.140900    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:14.195781 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:14.195752    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:21.076587 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.076549    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:21.130115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.130085    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:21.370740 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.370650    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"]
Jul 28 19:35:21.370988 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.370973    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy"
Jul 28 19:35:21.370988 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.370989    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy"
Jul 28 19:35:21.371115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.371002    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ece8b22-3f49-483f-b5ec-54eaae970199" containerName="revoke-keys"
Jul 28 19:35:21.371115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.371009    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ece8b22-3f49-483f-b5ec-54eaae970199" containerName="revoke-keys"
Jul 28 19:35:21.371115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.371059    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy"
Jul 28 19:35:21.371115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.371066    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ece8b22-3f49-483f-b5ec-54eaae970199" containerName="revoke-keys"
Jul 28 19:35:21.371115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.371073    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy"
Jul 28 19:35:21.374086 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.374067    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.376588 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.376566    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b965716e-openshift-default-dockercfg-zvnmj\""
Jul 28 19:35:21.385200 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.385178    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"]
Jul 28 19:35:21.390058 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.390040    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:21.455273 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.455235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-credential-socket\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.455478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.455288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-socket\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.455478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.455307    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/021e97db-3bc5-41e3-8d27-a774c40da483-istiod-ca-cert\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.455478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.455366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-certs\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.455478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.455458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-envoy\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.455716 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.455485    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbd7w\" (UniqueName: \"kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-kube-api-access-fbd7w\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.455716 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.455534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/021e97db-3bc5-41e3-8d27-a774c40da483-istio-podinfo\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.455716 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.455576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-istio-token\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.455716 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.455622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-data\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.556696 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.556659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-credential-socket\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.556870 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.556731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-socket\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.556870 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.556752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/021e97db-3bc5-41e3-8d27-a774c40da483-istiod-ca-cert\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.556870 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.556771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-certs\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.556870 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.556799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-envoy\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.556870 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.556829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbd7w\" (UniqueName: \"kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-kube-api-access-fbd7w\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.556870 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.556870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/021e97db-3bc5-41e3-8d27-a774c40da483-istio-podinfo\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.557145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.556893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-istio-token\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.557145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.556945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-data\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.557145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.557115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-credential-socket\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.557502 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.557476    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/021e97db-3bc5-41e3-8d27-a774c40da483-istiod-ca-cert\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.557585 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.557489    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-socket\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.557585 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.557520    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-certs\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.557655 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.557642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-data\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.559486 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.559463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-envoy\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.559706 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.559682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/021e97db-3bc5-41e3-8d27-a774c40da483-istio-podinfo\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.564572 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.564544    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-istio-token\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.564735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.564719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbd7w\" (UniqueName: \"kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-kube-api-access-fbd7w\") pod \"e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") " pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.684783 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.684684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:21.780109 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.780081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:21.820256 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.820232    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"]
Jul 28 19:35:21.822942 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:35:21.822902    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod021e97db_3bc5_41e3_8d27_a774c40da483.slice/crio-d18be0e5f83269b09f15bcfe3ba1c85ee0544485887473b8bb3b4caea0e643d4 WatchSource:0}: Error finding container d18be0e5f83269b09f15bcfe3ba1c85ee0544485887473b8bb3b4caea0e643d4: Status 404 returned error can't find the container with id d18be0e5f83269b09f15bcfe3ba1c85ee0544485887473b8bb3b4caea0e643d4
Jul 28 19:35:21.825119 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.825084    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:35:21.825199 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.825163    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:35:21.825252 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:21.825211    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:35:22.469372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:22.469337    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" event={"ID":"021e97db-3bc5-41e3-8d27-a774c40da483","Type":"ContainerStarted","Data":"5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b"}
Jul 28 19:35:22.469746 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:22.469387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" event={"ID":"021e97db-3bc5-41e3-8d27-a774c40da483","Type":"ContainerStarted","Data":"d18be0e5f83269b09f15bcfe3ba1c85ee0544485887473b8bb3b4caea0e643d4"}
Jul 28 19:35:22.488269 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:22.488217    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" podStartSLOduration=1.488201257 podStartE2EDuration="1.488201257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:22.487160744 +0000 UTC m=+1963.948768449" watchObservedRunningTime="2026-07-28 19:35:22.488201257 +0000 UTC m=+1963.949808960"
Jul 28 19:35:22.685835 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:22.685784    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:22.690702 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:22.690679    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:23.473968 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:23.473930    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:23.474849 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:23.474830    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:24.413203 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:24.413172    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:24.513288 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:24.513251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:24.670085 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:24.669992    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:24.970405 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:24.970312    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:25.234967 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:25.234847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:25.339430 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:25.339390    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:31.734337 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:31.734248    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:35:39.515154 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:39.515063    2563 patch_prober.go:28] interesting pod/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 19:35:39.515154 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:39.515127    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 19:35:40.556006 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:40.555961    2563 generic.go:356] "Generic (PLEG): container finished" podID="021e97db-3bc5-41e3-8d27-a774c40da483" containerID="5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b" exitCode=0
Jul 28 19:35:40.556459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:40.556036    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" event={"ID":"021e97db-3bc5-41e3-8d27-a774c40da483","Type":"ContainerDied","Data":"5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b"}
Jul 28 19:35:40.556459 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:40.556419    2563 scope.go:117] "RemoveContainer" containerID="5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b"
Jul 28 19:35:40.557428 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:40.557404    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:35:40.557505 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:40.557469    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:35:40.557505 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:40.557500    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:35:41.561957 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:41.561906    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" event={"ID":"021e97db-3bc5-41e3-8d27-a774c40da483","Type":"ContainerStarted","Data":"70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91"}
Jul 28 19:35:41.685035 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:41.685002    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:41.685299 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:41.685275    2563 patch_prober.go:28] interesting pod/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 28 19:35:41.685385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:41.685329    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 28 19:35:42.686069 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:42.686031    2563 patch_prober.go:28] interesting pod/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 28 19:35:42.686491 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:42.686085    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 28 19:35:43.685791 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:43.685754    2563 patch_prober.go:28] interesting pod/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 28 19:35:43.686000 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:43.685809    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 28 19:35:44.686069 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:44.686029    2563 patch_prober.go:28] interesting pod/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 28 19:35:44.686485 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:44.686085    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 28 19:35:45.688653 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:45.688623    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:45.689144 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:45.688899    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:45.689652 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:45.689637    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:35:49.411025 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:35:49.410988    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:01.145836 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.145790    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"]
Jul 28 19:36:01.146300 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.146121    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy"
Jul 28 19:36:01.146300 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.146133    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="05cee881-d937-45ed-99c9-17337af8d70b" containerName="istio-proxy"
Jul 28 19:36:01.149201 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.149184    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.153262 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.153245    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b965716e\"/\"e2e-lifecycle-model-b965716e-kserve-self-signed-certs\""
Jul 28 19:36:01.153372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.153245    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b965716e\"/\"default-dockercfg-h74d4\""
Jul 28 19:36:01.153372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.153286    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b965716e\"/\"kube-root-ca.crt\""
Jul 28 19:36:01.153372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.153249    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b965716e\"/\"openshift-service-ca.crt\""
Jul 28 19:36:01.157597 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.157575    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"]
Jul 28 19:36:01.171219 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.171189    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:01.186359 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.186335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-home\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.186630 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.186363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-dshm\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.186630 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.186380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af10da0f-132f-498d-baf0-000bf4163994-tls-certs\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.186630 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.186399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jphqx\" (UniqueName: \"kubernetes.io/projected/af10da0f-132f-498d-baf0-000bf4163994-kube-api-access-jphqx\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.186630 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.186445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-model-cache\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.186630 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.186476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-tmp-dir\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.287903 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.287863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-tmp-dir\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.288101 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.287996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-home\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.288101 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.288012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-dshm\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.288101 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.288027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af10da0f-132f-498d-baf0-000bf4163994-tls-certs\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.288101 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.288049    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jphqx\" (UniqueName: \"kubernetes.io/projected/af10da0f-132f-498d-baf0-000bf4163994-kube-api-access-jphqx\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.288101 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.288070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-model-cache\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.288363 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.288340    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-tmp-dir\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.288478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.288456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-home\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.288535 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.288481    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-model-cache\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.290416 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.290399    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-dshm\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.290662 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.290645    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af10da0f-132f-498d-baf0-000bf4163994-tls-certs\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.296273 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.296249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jphqx\" (UniqueName: \"kubernetes.io/projected/af10da0f-132f-498d-baf0-000bf4163994-kube-api-access-jphqx\") pod \"e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") " pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.458858 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.458767    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:01.594526 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.594493    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"]
Jul 28 19:36:01.597367 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:36:01.597337    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf10da0f_132f_498d_baf0_000bf4163994.slice/crio-450372ea3f682080db409e887e75258c66b3e371630babf4ac6640da45d98a8a WatchSource:0}: Error finding container 450372ea3f682080db409e887e75258c66b3e371630babf4ac6640da45d98a8a: Status 404 returned error can't find the container with id 450372ea3f682080db409e887e75258c66b3e371630babf4ac6640da45d98a8a
Jul 28 19:36:01.653267 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:01.653223    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9" event={"ID":"af10da0f-132f-498d-baf0-000bf4163994","Type":"ContainerStarted","Data":"450372ea3f682080db409e887e75258c66b3e371630babf4ac6640da45d98a8a"}
Jul 28 19:36:02.658498 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:02.658456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9" event={"ID":"af10da0f-132f-498d-baf0-000bf4163994","Type":"ContainerStarted","Data":"af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01"}
Jul 28 19:36:02.677257 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:02.677213    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9" podStartSLOduration=1.677198376 podStartE2EDuration="1.677198376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:02.676580636 +0000 UTC m=+2004.138188342" watchObservedRunningTime="2026-07-28 19:36:02.677198376 +0000 UTC m=+2004.138806079"
Jul 28 19:36:03.628610 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:03.628565    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:04.635951 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:04.635902    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:36:11.459039 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:11.458983    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:11.459556 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:11.459058    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:11.471492 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:11.471468    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:11.708683 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:11.708655    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:36:29.666620 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:29.666580    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:36:33.867883 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:33.867847    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"]
Jul 28 19:36:33.871347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:33.871318    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"
Jul 28 19:36:33.873913 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:33.873893    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:36:33.873913 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:33.873903    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:36:33.877238 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:33.877214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"]
Jul 28 19:36:33.980255 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:33.980220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8adb01b6-fcce-4eaf-a8f5-d4ef29df925e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b965716e-hljt9\" (UID: \"8adb01b6-fcce-4eaf-a8f5-d4ef29df925e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"
Jul 28 19:36:34.080727 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:34.080693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8adb01b6-fcce-4eaf-a8f5-d4ef29df925e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b965716e-hljt9\" (UID: \"8adb01b6-fcce-4eaf-a8f5-d4ef29df925e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"
Jul 28 19:36:34.081394 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:34.081373    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8adb01b6-fcce-4eaf-a8f5-d4ef29df925e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b965716e-hljt9\" (UID: \"8adb01b6-fcce-4eaf-a8f5-d4ef29df925e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"
Jul 28 19:36:34.183870 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:34.183794    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"
Jul 28 19:36:34.317802 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:34.317775    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"]
Jul 28 19:36:34.320331 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:36:34.320300    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8adb01b6_fcce_4eaf_a8f5_d4ef29df925e.slice/crio-0bd010a3e7e8d709dfcb08f9a7c259ef1ddd8fdbe3f2e86c4e00b07144117b3c WatchSource:0}: Error finding container 0bd010a3e7e8d709dfcb08f9a7c259ef1ddd8fdbe3f2e86c4e00b07144117b3c: Status 404 returned error can't find the container with id 0bd010a3e7e8d709dfcb08f9a7c259ef1ddd8fdbe3f2e86c4e00b07144117b3c
Jul 28 19:36:34.802607 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:34.802517    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9" event={"ID":"8adb01b6-fcce-4eaf-a8f5-d4ef29df925e","Type":"ContainerStarted","Data":"f8ce8e6e8b08e747a4d5088988a4859caa6b53591ac721c0320f4b64e27533e9"}
Jul 28 19:36:34.802607 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:34.802565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9" event={"ID":"8adb01b6-fcce-4eaf-a8f5-d4ef29df925e","Type":"ContainerStarted","Data":"0bd010a3e7e8d709dfcb08f9a7c259ef1ddd8fdbe3f2e86c4e00b07144117b3c"}
Jul 28 19:36:34.820935 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:34.820881    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9" podStartSLOduration=1.82086666 podStartE2EDuration="1.82086666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:34.819243033 +0000 UTC m=+2036.280850735" watchObservedRunningTime="2026-07-28 19:36:34.82086666 +0000 UTC m=+2036.282474381"
Jul 28 19:36:35.808239 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:35.808209    2563 generic.go:356] "Generic (PLEG): container finished" podID="8adb01b6-fcce-4eaf-a8f5-d4ef29df925e" containerID="f8ce8e6e8b08e747a4d5088988a4859caa6b53591ac721c0320f4b64e27533e9" exitCode=0
Jul 28 19:36:35.808630 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:35.808296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9" event={"ID":"8adb01b6-fcce-4eaf-a8f5-d4ef29df925e","Type":"ContainerDied","Data":"f8ce8e6e8b08e747a4d5088988a4859caa6b53591ac721c0320f4b64e27533e9"}
Jul 28 19:36:36.943292 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:36.943264    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"
Jul 28 19:36:37.106338 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:37.106302    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8adb01b6-fcce-4eaf-a8f5-d4ef29df925e-maas-api-service-ca\") pod \"8adb01b6-fcce-4eaf-a8f5-d4ef29df925e\" (UID: \"8adb01b6-fcce-4eaf-a8f5-d4ef29df925e\") "
Jul 28 19:36:37.106729 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:37.106697    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8adb01b6-fcce-4eaf-a8f5-d4ef29df925e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8adb01b6-fcce-4eaf-a8f5-d4ef29df925e" (UID: "8adb01b6-fcce-4eaf-a8f5-d4ef29df925e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:37.207774 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:37.207733    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8adb01b6-fcce-4eaf-a8f5-d4ef29df925e-maas-api-service-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:36:37.472069 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:37.471965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:37.818682 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:37.818648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9" event={"ID":"8adb01b6-fcce-4eaf-a8f5-d4ef29df925e","Type":"ContainerDied","Data":"0bd010a3e7e8d709dfcb08f9a7c259ef1ddd8fdbe3f2e86c4e00b07144117b3c"}
Jul 28 19:36:37.818829 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:37.818697    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bd010a3e7e8d709dfcb08f9a7c259ef1ddd8fdbe3f2e86c4e00b07144117b3c"
Jul 28 19:36:37.818829 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:37.818663    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"
Jul 28 19:36:39.164253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:39.164221    2563 scope.go:117] "RemoveContainer" containerID="02c5c05810d1d18d2e4255e764ac62a20f45ee7040123b3a46bdf04b2adea668"
Jul 28 19:36:47.943152 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:47.943108    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:36:49.309540 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:49.309507    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:49.409159 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:49.409122    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:49.473997 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:49.473956    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"]
Jul 28 19:36:49.476457 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:49.476432    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b965716e-hljt9"]
Jul 28 19:36:49.477007 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:49.476969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:49.527746 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:49.527702    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:51.144061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:51.144026    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8adb01b6-fcce-4eaf-a8f5-d4ef29df925e" path="/var/lib/kubelet/pods/8adb01b6-fcce-4eaf-a8f5-d4ef29df925e/volumes"
Jul 28 19:36:52.595080 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:52.595040    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:56.251724 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:56.251685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:36:57.570630 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:57.570598    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"]
Jul 28 19:36:57.608732 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:36:57.608697    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:12.590229 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.590123    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9" podUID="af10da0f-132f-498d-baf0-000bf4163994" containerName="main" containerID="cri-o://af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01" gracePeriod=45
Jul 28 19:37:12.840754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.840701    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:37:12.907859 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.907818    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-home\") pod \"af10da0f-132f-498d-baf0-000bf4163994\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") "
Jul 28 19:37:12.907859 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.907860    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jphqx\" (UniqueName: \"kubernetes.io/projected/af10da0f-132f-498d-baf0-000bf4163994-kube-api-access-jphqx\") pod \"af10da0f-132f-498d-baf0-000bf4163994\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") "
Jul 28 19:37:12.908122 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.907890    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-dshm\") pod \"af10da0f-132f-498d-baf0-000bf4163994\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") "
Jul 28 19:37:12.908122 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.907965    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-tmp-dir\") pod \"af10da0f-132f-498d-baf0-000bf4163994\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") "
Jul 28 19:37:12.908122 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.908013    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-model-cache\") pod \"af10da0f-132f-498d-baf0-000bf4163994\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") "
Jul 28 19:37:12.908122 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.908058    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af10da0f-132f-498d-baf0-000bf4163994-tls-certs\") pod \"af10da0f-132f-498d-baf0-000bf4163994\" (UID: \"af10da0f-132f-498d-baf0-000bf4163994\") "
Jul 28 19:37:12.908323 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.908163    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-home" (OuterVolumeSpecName: "home") pod "af10da0f-132f-498d-baf0-000bf4163994" (UID: "af10da0f-132f-498d-baf0-000bf4163994"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:12.908323 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.908309    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-home\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:12.908408 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.908299    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "af10da0f-132f-498d-baf0-000bf4163994" (UID: "af10da0f-132f-498d-baf0-000bf4163994"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:12.908408 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.908316    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-model-cache" (OuterVolumeSpecName: "model-cache") pod "af10da0f-132f-498d-baf0-000bf4163994" (UID: "af10da0f-132f-498d-baf0-000bf4163994"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:12.910244 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.910212    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af10da0f-132f-498d-baf0-000bf4163994-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af10da0f-132f-498d-baf0-000bf4163994" (UID: "af10da0f-132f-498d-baf0-000bf4163994"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:37:12.910349 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.910249    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-dshm" (OuterVolumeSpecName: "dshm") pod "af10da0f-132f-498d-baf0-000bf4163994" (UID: "af10da0f-132f-498d-baf0-000bf4163994"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:12.910349 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.910284    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af10da0f-132f-498d-baf0-000bf4163994-kube-api-access-jphqx" (OuterVolumeSpecName: "kube-api-access-jphqx") pod "af10da0f-132f-498d-baf0-000bf4163994" (UID: "af10da0f-132f-498d-baf0-000bf4163994"). InnerVolumeSpecName "kube-api-access-jphqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:37:12.972755 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.972719    2563 generic.go:356] "Generic (PLEG): container finished" podID="af10da0f-132f-498d-baf0-000bf4163994" containerID="af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01" exitCode=0
Jul 28 19:37:12.972947 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.972762    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"
Jul 28 19:37:12.972947 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.972807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9" event={"ID":"af10da0f-132f-498d-baf0-000bf4163994","Type":"ContainerDied","Data":"af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01"}
Jul 28 19:37:12.972947 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.972849    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9" event={"ID":"af10da0f-132f-498d-baf0-000bf4163994","Type":"ContainerDied","Data":"450372ea3f682080db409e887e75258c66b3e371630babf4ac6640da45d98a8a"}
Jul 28 19:37:12.972947 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.972867    2563 scope.go:117] "RemoveContainer" containerID="af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01"
Jul 28 19:37:12.981641 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.981625    2563 scope.go:117] "RemoveContainer" containerID="af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01"
Jul 28 19:37:12.982054 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:37:12.982033    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01\": container with ID starting with af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01 not found: ID does not exist" containerID="af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01"
Jul 28 19:37:12.982100 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.982064    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01"} err="failed to get container status \"af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01\": rpc error: code = NotFound desc = could not find container \"af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01\": container with ID starting with af24c63dbbf51bdda8fb12402bf46abbb588fb0d802dfe91a563d521d93b9a01 not found: ID does not exist"
Jul 28 19:37:12.992347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.992322    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"]
Jul 28 19:37:12.995639 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:12.995618    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-b965716e/e2e-lifecycle-model-b965716e-kserve-5b6f8ffff9-k7hj9"]
Jul 28 19:37:13.009581 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:13.009559    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jphqx\" (UniqueName: \"kubernetes.io/projected/af10da0f-132f-498d-baf0-000bf4163994-kube-api-access-jphqx\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:13.009581 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:13.009581    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-dshm\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:13.009723 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:13.009593    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-tmp-dir\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:13.009723 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:13.009603    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af10da0f-132f-498d-baf0-000bf4163994-model-cache\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:13.009723 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:13.009613    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af10da0f-132f-498d-baf0-000bf4163994-tls-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:13.143947 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:13.143844    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af10da0f-132f-498d-baf0-000bf4163994" path="/var/lib/kubelet/pods/af10da0f-132f-498d-baf0-000bf4163994/volumes"
Jul 28 19:37:34.279347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:34.279298    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:34.612234 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:34.612202    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:39.086768 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:39.086730    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:37:39.091794 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:39.091775    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:37:39.183764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:39.183732    2563 scope.go:117] "RemoveContainer" containerID="987dc2eb56f8b21a87d610de1a2865e99e585fcd20db8328384be2d1e0fa7e57"
Jul 28 19:37:39.191785 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:39.191763    2563 scope.go:117] "RemoveContainer" containerID="2bc2507bbf02eefb2ba0163f767e8b81dabd7185d4384e62e04f1e6717f4b480"
Jul 28 19:37:40.213861 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:40.213823    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:45.645232 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:45.645181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:45.704721 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:45.704686    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:45.888953 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:45.888901    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"]
Jul 28 19:37:45.889195 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:45.889164    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy" containerID="cri-o://70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91" gracePeriod=30
Jul 28 19:37:45.909189 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:45.909098    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:46.125769 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:46.125723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:47.182052 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:47.181990    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:47.273727 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:47.273664    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:47.368432 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:47.368401    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:48.062116 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:48.062082    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:48.160990 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:48.160955    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:48.260616 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:48.260581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:49.446557 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:49.446521    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:37:49.572490 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:49.572451    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:51.140641 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.140619    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:37:51.155949 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.155857    2563 generic.go:356] "Generic (PLEG): container finished" podID="021e97db-3bc5-41e3-8d27-a774c40da483" containerID="70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91" exitCode=0
Jul 28 19:37:51.155949 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.155934    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"
Jul 28 19:37:51.156145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.155956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" event={"ID":"021e97db-3bc5-41e3-8d27-a774c40da483","Type":"ContainerDied","Data":"70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91"}
Jul 28 19:37:51.156145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.156019    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2" event={"ID":"021e97db-3bc5-41e3-8d27-a774c40da483","Type":"ContainerDied","Data":"d18be0e5f83269b09f15bcfe3ba1c85ee0544485887473b8bb3b4caea0e643d4"}
Jul 28 19:37:51.156145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.156052    2563 scope.go:117] "RemoveContainer" containerID="70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91"
Jul 28 19:37:51.165582 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.165560    2563 scope.go:117] "RemoveContainer" containerID="5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b"
Jul 28 19:37:51.173632 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.173611    2563 scope.go:117] "RemoveContainer" containerID="70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91"
Jul 28 19:37:51.173908 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:37:51.173888    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91\": container with ID starting with 70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91 not found: ID does not exist" containerID="70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91"
Jul 28 19:37:51.174129 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.173935    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91"} err="failed to get container status \"70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91\": rpc error: code = NotFound desc = could not find container \"70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91\": container with ID starting with 70e4ab1b25c03501022904b764447a820ab4977ce3f1495902edf5bcce985e91 not found: ID does not exist"
Jul 28 19:37:51.174129 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.173953    2563 scope.go:117] "RemoveContainer" containerID="5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b"
Jul 28 19:37:51.174256 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:37:51.174141    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b\": container with ID starting with 5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b not found: ID does not exist" containerID="5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b"
Jul 28 19:37:51.174256 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.174158    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b"} err="failed to get container status \"5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b\": rpc error: code = NotFound desc = could not find container \"5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b\": container with ID starting with 5fd7b33e140c432384085c8deb43294f1f6d2063bda1ef7e7ea88b6a4811405b not found: ID does not exist"
Jul 28 19:37:51.228358 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228326    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-certs\") pod \"021e97db-3bc5-41e3-8d27-a774c40da483\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") "
Jul 28 19:37:51.228534 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228382    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-data\") pod \"021e97db-3bc5-41e3-8d27-a774c40da483\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") "
Jul 28 19:37:51.228534 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228416    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbd7w\" (UniqueName: \"kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-kube-api-access-fbd7w\") pod \"021e97db-3bc5-41e3-8d27-a774c40da483\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") "
Jul 28 19:37:51.228534 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228434    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/021e97db-3bc5-41e3-8d27-a774c40da483-istiod-ca-cert\") pod \"021e97db-3bc5-41e3-8d27-a774c40da483\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") "
Jul 28 19:37:51.228534 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228463    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/021e97db-3bc5-41e3-8d27-a774c40da483-istio-podinfo\") pod \"021e97db-3bc5-41e3-8d27-a774c40da483\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") "
Jul 28 19:37:51.228534 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228506    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-envoy\") pod \"021e97db-3bc5-41e3-8d27-a774c40da483\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") "
Jul 28 19:37:51.228796 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228617    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-istio-token\") pod \"021e97db-3bc5-41e3-8d27-a774c40da483\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") "
Jul 28 19:37:51.228796 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228653    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-credential-socket\") pod \"021e97db-3bc5-41e3-8d27-a774c40da483\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") "
Jul 28 19:37:51.228796 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228629    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "021e97db-3bc5-41e3-8d27-a774c40da483" (UID: "021e97db-3bc5-41e3-8d27-a774c40da483"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:51.228796 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228761    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-socket\") pod \"021e97db-3bc5-41e3-8d27-a774c40da483\" (UID: \"021e97db-3bc5-41e3-8d27-a774c40da483\") "
Jul 28 19:37:51.229032 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.228969    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "021e97db-3bc5-41e3-8d27-a774c40da483" (UID: "021e97db-3bc5-41e3-8d27-a774c40da483"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:51.229032 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.229019    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021e97db-3bc5-41e3-8d27-a774c40da483-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "021e97db-3bc5-41e3-8d27-a774c40da483" (UID: "021e97db-3bc5-41e3-8d27-a774c40da483"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:37:51.229142 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.229029    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:51.229142 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.229125    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "021e97db-3bc5-41e3-8d27-a774c40da483" (UID: "021e97db-3bc5-41e3-8d27-a774c40da483"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:51.229356 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.229314    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-data" (OuterVolumeSpecName: "istio-data") pod "021e97db-3bc5-41e3-8d27-a774c40da483" (UID: "021e97db-3bc5-41e3-8d27-a774c40da483"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:51.230863 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.230830    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/021e97db-3bc5-41e3-8d27-a774c40da483-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "021e97db-3bc5-41e3-8d27-a774c40da483" (UID: "021e97db-3bc5-41e3-8d27-a774c40da483"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:37:51.230995 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.230903    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-kube-api-access-fbd7w" (OuterVolumeSpecName: "kube-api-access-fbd7w") pod "021e97db-3bc5-41e3-8d27-a774c40da483" (UID: "021e97db-3bc5-41e3-8d27-a774c40da483"). InnerVolumeSpecName "kube-api-access-fbd7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:37:51.231050 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.230994    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "021e97db-3bc5-41e3-8d27-a774c40da483" (UID: "021e97db-3bc5-41e3-8d27-a774c40da483"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:37:51.231257 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.231240    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-istio-token" (OuterVolumeSpecName: "istio-token") pod "021e97db-3bc5-41e3-8d27-a774c40da483" (UID: "021e97db-3bc5-41e3-8d27-a774c40da483"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:37:51.329497 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.329458    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:51.329497 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.329490    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:51.329497 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.329502    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:51.329736 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.329513    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:51.329736 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.329524    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/021e97db-3bc5-41e3-8d27-a774c40da483-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:51.329736 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.329536    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbd7w\" (UniqueName: \"kubernetes.io/projected/021e97db-3bc5-41e3-8d27-a774c40da483-kube-api-access-fbd7w\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:51.329736 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.329546    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/021e97db-3bc5-41e3-8d27-a774c40da483-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:51.329736 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.329555    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/021e97db-3bc5-41e3-8d27-a774c40da483-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:37:51.480389 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.480353    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"]
Jul 28 19:37:51.483937 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:51.483889    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b965716e-openshift-default-69c9588dcb-cwrf2"]
Jul 28 19:37:53.142961 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:53.142907    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" path="/var/lib/kubelet/pods/021e97db-3bc5-41e3-8d27-a774c40da483/volumes"
Jul 28 19:37:55.422240 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:55.422203    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:55.483642 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:55.483594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:56.045698 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:37:56.045663    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:01.444965 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:01.444931    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:38:02.168126 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:02.168094    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:02.274268 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:02.274232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:02.371562 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:02.371522    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:12.838876 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:12.838837    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:13.173401 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:13.173289    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:17.484092 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:17.484057    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:34.109949 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:34.109822    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:39.214781 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:39.214747    2563 scope.go:117] "RemoveContainer" containerID="7328d13ff7491c15171e040b5c91fef68f74ddab5fc1cb726e72f289e8ab8aba"
Jul 28 19:38:40.265783 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.265732    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:40.367658 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.367623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:40.614358 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.614326    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:40.891355 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891258    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"]
Jul 28 19:38:40.891597 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891583    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af10da0f-132f-498d-baf0-000bf4163994" containerName="main"
Jul 28 19:38:40.891640 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891599    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="af10da0f-132f-498d-baf0-000bf4163994" containerName="main"
Jul 28 19:38:40.891640 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891613    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8adb01b6-fcce-4eaf-a8f5-d4ef29df925e" containerName="revoke-keys"
Jul 28 19:38:40.891640 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891619    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8adb01b6-fcce-4eaf-a8f5-d4ef29df925e" containerName="revoke-keys"
Jul 28 19:38:40.891640 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891628    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy"
Jul 28 19:38:40.891640 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891634    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy"
Jul 28 19:38:40.891824 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891645    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy"
Jul 28 19:38:40.891824 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891653    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy"
Jul 28 19:38:40.891824 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891723    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy"
Jul 28 19:38:40.891824 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891733    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8adb01b6-fcce-4eaf-a8f5-d4ef29df925e" containerName="revoke-keys"
Jul 28 19:38:40.891824 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.891741    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="af10da0f-132f-498d-baf0-000bf4163994" containerName="main"
Jul 28 19:38:40.895726 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.895709    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:40.898349 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.898322    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e55e7d-openshift-default-dockercfg-kkc7t\""
Jul 28 19:38:40.906533 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.906501    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"]
Jul 28 19:38:40.912734 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:40.912702    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:41.044397 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.044352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-credential-socket\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.044397 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.044403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-token\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.044641 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.044426    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-certs\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.044641 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.044445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-data\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.044641 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.044500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-envoy\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.044641 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.044556    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb7286e-2899-4a02-86dc-baacbf931eb2-istiod-ca-cert\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.044641 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.044607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-podinfo\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.044641 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.044636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-socket\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.044833 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.044665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwff5\" (UniqueName: \"kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-kube-api-access-kwff5\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.145975 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.145871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-credential-socket\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.145975 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.145905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-token\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.145975 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.145957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-certs\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.145975 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.145974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-data\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.146274 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.145996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-envoy\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.146274 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.146096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb7286e-2899-4a02-86dc-baacbf931eb2-istiod-ca-cert\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.146274 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.146141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-podinfo\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.146274 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.146184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-socket\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.146274 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.146226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwff5\" (UniqueName: \"kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-kube-api-access-kwff5\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.146557 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.146415    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-credential-socket\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.146900 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.146753    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-certs\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.146900 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.146857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-data\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.146900 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.146868    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-socket\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.147173 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.147145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb7286e-2899-4a02-86dc-baacbf931eb2-istiod-ca-cert\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.148635 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.148614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-envoy\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.149100 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.149082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-podinfo\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.154288 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.154265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwff5\" (UniqueName: \"kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-kube-api-access-kwff5\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.154687 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.154667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-token\") pod \"e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") " pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.194008 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.193969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:41.207986 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.207955    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:41.347633 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.347609    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"]
Jul 28 19:38:41.350196 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:38:41.350169    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb7286e_2899_4a02_86dc_baacbf931eb2.slice/crio-c51226684cd8dfa8d2877297d9940cb2cf896191778602ced1dbf59e656ea32f WatchSource:0}: Error finding container c51226684cd8dfa8d2877297d9940cb2cf896191778602ced1dbf59e656ea32f: Status 404 returned error can't find the container with id c51226684cd8dfa8d2877297d9940cb2cf896191778602ced1dbf59e656ea32f
Jul 28 19:38:41.352253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.352222    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:38:41.352337 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.352311    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:38:41.352377 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.352339    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:38:41.366969 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:41.366943    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" event={"ID":"fcb7286e-2899-4a02-86dc-baacbf931eb2","Type":"ContainerStarted","Data":"c51226684cd8dfa8d2877297d9940cb2cf896191778602ced1dbf59e656ea32f"}
Jul 28 19:38:42.372267 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:42.372233    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" event={"ID":"fcb7286e-2899-4a02-86dc-baacbf931eb2","Type":"ContainerStarted","Data":"2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8"}
Jul 28 19:38:42.391738 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:42.391685    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" podStartSLOduration=2.391668854 podStartE2EDuration="2.391668854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:38:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:42.390888193 +0000 UTC m=+2163.852495922" watchObservedRunningTime="2026-07-28 19:38:42.391668854 +0000 UTC m=+2163.853276611"
Jul 28 19:38:43.209038 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:43.209002    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:43.213631 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:43.213608    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:43.376675 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:43.376648    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:43.377552 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:43.377532    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:38:44.313589 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:44.313552    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:44.399964 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:44.399932    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:44.462237 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:44.462202    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:45.091421 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:45.091387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:45.192013 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:45.191965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:45.259789 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:45.259754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:49.094717 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:49.094683    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:50.044354 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:50.044305    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:38:58.378204 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:58.378166    2563 patch_prober.go:28] interesting pod/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 28 19:38:58.378619 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:58.378237    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 28 19:38:58.445793 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:58.445763    2563 generic.go:356] "Generic (PLEG): container finished" podID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerID="2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8" exitCode=0
Jul 28 19:38:58.445980 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:58.445838    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" event={"ID":"fcb7286e-2899-4a02-86dc-baacbf931eb2","Type":"ContainerDied","Data":"2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8"}
Jul 28 19:38:58.446253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:58.446239    2563 scope.go:117] "RemoveContainer" containerID="2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8"
Jul 28 19:38:58.447194 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:58.447167    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:38:58.447285 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:58.447248    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:38:58.447355 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:58.447297    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:38:59.451510 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:38:59.451474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" event={"ID":"fcb7286e-2899-4a02-86dc-baacbf931eb2","Type":"ContainerStarted","Data":"e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4"}
Jul 28 19:39:00.208725 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:00.208687    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:39:00.209173 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:00.209146    2563 patch_prober.go:28] interesting pod/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 28 19:39:00.209376 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:00.209351    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 28 19:39:01.208860 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:01.208807    2563 patch_prober.go:28] interesting pod/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 28 19:39:01.209392 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:01.208897    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 28 19:39:02.209217 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:02.209177    2563 patch_prober.go:28] interesting pod/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 28 19:39:02.209835 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:02.209802    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 28 19:39:03.208534 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:03.208492    2563 patch_prober.go:28] interesting pod/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 28 19:39:03.208747 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:03.208560    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 28 19:39:04.212913 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:04.212881    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:39:04.213408 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:04.213271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:39:04.213865 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:04.213847    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:39:09.263548 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.263507    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:09.585443 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.585390    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:09.848600 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.848507    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"]
Jul 28 19:39:09.848964 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.848946    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="021e97db-3bc5-41e3-8d27-a774c40da483" containerName="istio-proxy"
Jul 28 19:39:09.851994 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.851972    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:09.854837 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.854815    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5f6911-openshift-default-dockercfg-5stjb\""
Jul 28 19:39:09.863678 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.863649    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"]
Jul 28 19:39:09.878172 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.878131    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:09.989859 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.989817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-token\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:09.989859 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.989866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istiod-ca-cert\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:09.990119 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.989940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-credential-socket\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:09.990119 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.989969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-socket\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:09.990119 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.990003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-certs\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:09.990119 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.990022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85rpw\" (UniqueName: \"kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-kube-api-access-85rpw\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:09.990119 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.990049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-data\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:09.990119 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.990066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-envoy\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:09.990319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:09.990119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-podinfo\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.090949 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.090887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-socket\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.090949 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.090951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-certs\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091206 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.090984    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rpw\" (UniqueName: \"kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-kube-api-access-85rpw\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091206 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-data\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091206 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-envoy\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091206 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-podinfo\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091206 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-token\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091206 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istiod-ca-cert\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091206 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-credential-socket\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091578 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-data\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091578 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-certs\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091578 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-socket\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091578 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-credential-socket\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.091901 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.091884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istiod-ca-cert\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.093884 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.093857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-envoy\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.094048 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.094028    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-podinfo\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.098966 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.098860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-token\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.099116 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.099093    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85rpw\" (UniqueName: \"kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-kube-api-access-85rpw\") pod \"e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") " pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.136119 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.136088    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:10.162814 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.162784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:10.316225 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.316171    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"]
Jul 28 19:39:10.319115 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:39:10.319082    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded4ba6b5_0f98_4501_8e9e_3790984e243f.slice/crio-ef758079e623dd237732c34665d4f34b9ebad6d47a110d20593db90e3d4c9faa WatchSource:0}: Error finding container ef758079e623dd237732c34665d4f34b9ebad6d47a110d20593db90e3d4c9faa: Status 404 returned error can't find the container with id ef758079e623dd237732c34665d4f34b9ebad6d47a110d20593db90e3d4c9faa
Jul 28 19:39:10.321257 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.321223    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:39:10.321325 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.321294    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:39:10.321370 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.321342    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:39:10.500595 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.500559    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" event={"ID":"ed4ba6b5-0f98-4501-8e9e-3790984e243f","Type":"ContainerStarted","Data":"873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e"}
Jul 28 19:39:10.500789 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.500623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" event={"ID":"ed4ba6b5-0f98-4501-8e9e-3790984e243f","Type":"ContainerStarted","Data":"ef758079e623dd237732c34665d4f34b9ebad6d47a110d20593db90e3d4c9faa"}
Jul 28 19:39:10.519171 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:10.519107    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" podStartSLOduration=1.519086027 podStartE2EDuration="1.519086027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:39:10.517728826 +0000 UTC m=+2191.979336528" watchObservedRunningTime="2026-07-28 19:39:10.519086027 +0000 UTC m=+2191.980693731"
Jul 28 19:39:11.163528 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:11.163486    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:11.168506 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:11.168479    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:11.505352 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:11.505274    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:11.506376 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:11.506360    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:12.383209 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:12.383168    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:12.483212 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:12.483163    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:12.545656 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:12.545623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:13.079418 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:13.079381    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:13.165370 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:13.165334    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:19.440294 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:19.440250    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:39:27.531599 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:27.531559    2563 patch_prober.go:28] interesting pod/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 19:39:27.532049 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:27.531642    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 19:39:28.586346 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:28.586308    2563 generic.go:356] "Generic (PLEG): container finished" podID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerID="873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e" exitCode=0
Jul 28 19:39:28.586780 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:28.586383    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" event={"ID":"ed4ba6b5-0f98-4501-8e9e-3790984e243f","Type":"ContainerDied","Data":"873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e"}
Jul 28 19:39:28.586780 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:28.586766    2563 scope.go:117] "RemoveContainer" containerID="873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e"
Jul 28 19:39:28.587717 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:28.587691    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:39:28.587805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:28.587754    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:39:28.587805 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:28.587783    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:39:29.593545 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:29.593503    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" event={"ID":"ed4ba6b5-0f98-4501-8e9e-3790984e243f","Type":"ContainerStarted","Data":"af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1"}
Jul 28 19:39:30.162970 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:30.162935    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:30.163144 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:30.163072    2563 patch_prober.go:28] interesting pod/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 28 19:39:30.163144 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:30.163113    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 28 19:39:31.164112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:31.164072    2563 patch_prober.go:28] interesting pod/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 28 19:39:31.164504 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:31.164128    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 28 19:39:32.163677 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:32.163635    2563 patch_prober.go:28] interesting pod/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 28 19:39:32.163861 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:32.163694    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 28 19:39:33.164032 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:33.163987    2563 patch_prober.go:28] interesting pod/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 28 19:39:33.164411 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:33.164046    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 28 19:39:34.167351 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:34.167311    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:34.167721 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:34.167588    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:34.168244 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:34.168225    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:39:37.347175 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:37.347141    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:56.030184 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:56.030145    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:59.444580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:39:59.444490    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:40:10.241874 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:10.241835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:40:32.942999 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:32.942966    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"]
Jul 28 19:40:32.946254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:32.946236    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:32.950478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:32.950457    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5f6911\"/\"openshift-service-ca.crt\""
Jul 28 19:40:32.950599 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:32.950516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5f6911\"/\"default-dockercfg-zwlnd\""
Jul 28 19:40:32.950599 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:32.950516    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5f6911\"/\"kube-root-ca.crt\""
Jul 28 19:40:32.950716 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:32.950516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5f6911\"/\"test-model-5f6911-kserve-self-signed-certs\""
Jul 28 19:40:32.954969 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:32.954948    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"]
Jul 28 19:40:32.970912 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:32.970890    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:33.012284 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.012252    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-tmp-dir\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.012433 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.012318    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c85d384e-b1c4-41a4-af96-acaf10b3b554-tls-certs\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.012433 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.012394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-dshm\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.012503 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.012431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k9l4\" (UniqueName: \"kubernetes.io/projected/c85d384e-b1c4-41a4-af96-acaf10b3b554-kube-api-access-9k9l4\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.012503 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.012454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-home\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.012503 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.012469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-model-cache\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.113236 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.113194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-dshm\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.113236 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.113247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k9l4\" (UniqueName: \"kubernetes.io/projected/c85d384e-b1c4-41a4-af96-acaf10b3b554-kube-api-access-9k9l4\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.113509 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.113267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-home\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.113509 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.113282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-model-cache\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.113509 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.113308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-tmp-dir\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.113509 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.113355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c85d384e-b1c4-41a4-af96-acaf10b3b554-tls-certs\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.113705 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.113663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-model-cache\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.113767 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.113741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-home\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.113823 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.113792    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-tmp-dir\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.115735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.115702    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-dshm\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.115983 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.115965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c85d384e-b1c4-41a4-af96-acaf10b3b554-tls-certs\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.121185 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.121167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k9l4\" (UniqueName: \"kubernetes.io/projected/c85d384e-b1c4-41a4-af96-acaf10b3b554-kube-api-access-9k9l4\") pod \"test-model-5f6911-kserve-76f59dbbcc-wbz7d\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") " pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.256435 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.256340    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:33.414468 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.414439    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"]
Jul 28 19:40:33.417154 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:40:33.417130    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc85d384e_b1c4_41a4_af96_acaf10b3b554.slice/crio-143cfde38933a85426b1572d9e9b42818525dabb6a305f93b7f9831a795453ff WatchSource:0}: Error finding container 143cfde38933a85426b1572d9e9b42818525dabb6a305f93b7f9831a795453ff: Status 404 returned error can't find the container with id 143cfde38933a85426b1572d9e9b42818525dabb6a305f93b7f9831a795453ff
Jul 28 19:40:33.892133 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.892097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d" event={"ID":"c85d384e-b1c4-41a4-af96-acaf10b3b554","Type":"ContainerStarted","Data":"112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4"}
Jul 28 19:40:33.892337 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.892147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d" event={"ID":"c85d384e-b1c4-41a4-af96-acaf10b3b554","Type":"ContainerStarted","Data":"143cfde38933a85426b1572d9e9b42818525dabb6a305f93b7f9831a795453ff"}
Jul 28 19:40:33.914824 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:33.914768    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d" podStartSLOduration=1.914750552 podStartE2EDuration="1.914750552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:33.90697176 +0000 UTC m=+2275.368579466" watchObservedRunningTime="2026-07-28 19:40:33.914750552 +0000 UTC m=+2275.376358255"
Jul 28 19:40:38.438367 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:38.438331    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:40:43.257176 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:43.257135    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:43.257594 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:43.257191    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:43.269994 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:43.269970    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:40:43.945932 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:40:43.945883    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:41:22.344114 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:22.344078    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:23.254141 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:23.254112    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"]
Jul 28 19:41:23.265669 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:23.265641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:27.042458 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:27.042427    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:41:34.161098 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:34.161011    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:34.224135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:34.224100    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:34.278829 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:34.278799    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:34.340645 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:34.340597    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:34.723790 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:34.723759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:38.273570 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.273530    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d" podUID="c85d384e-b1c4-41a4-af96-acaf10b3b554" containerName="main" containerID="cri-o://112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4" gracePeriod=45
Jul 28 19:41:38.524539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.524476    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:41:38.582938 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.582889    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k9l4\" (UniqueName: \"kubernetes.io/projected/c85d384e-b1c4-41a4-af96-acaf10b3b554-kube-api-access-9k9l4\") pod \"c85d384e-b1c4-41a4-af96-acaf10b3b554\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") "
Jul 28 19:41:38.583135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.582982    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c85d384e-b1c4-41a4-af96-acaf10b3b554-tls-certs\") pod \"c85d384e-b1c4-41a4-af96-acaf10b3b554\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") "
Jul 28 19:41:38.583135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.583028    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-home\") pod \"c85d384e-b1c4-41a4-af96-acaf10b3b554\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") "
Jul 28 19:41:38.583135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.583105    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-model-cache\") pod \"c85d384e-b1c4-41a4-af96-acaf10b3b554\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") "
Jul 28 19:41:38.583319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.583143    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-dshm\") pod \"c85d384e-b1c4-41a4-af96-acaf10b3b554\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") "
Jul 28 19:41:38.583319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.583168    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-tmp-dir\") pod \"c85d384e-b1c4-41a4-af96-acaf10b3b554\" (UID: \"c85d384e-b1c4-41a4-af96-acaf10b3b554\") "
Jul 28 19:41:38.583432 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.583312    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-home" (OuterVolumeSpecName: "home") pod "c85d384e-b1c4-41a4-af96-acaf10b3b554" (UID: "c85d384e-b1c4-41a4-af96-acaf10b3b554"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:38.583485 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.583426    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-model-cache" (OuterVolumeSpecName: "model-cache") pod "c85d384e-b1c4-41a4-af96-acaf10b3b554" (UID: "c85d384e-b1c4-41a4-af96-acaf10b3b554"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:38.583485 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.583474    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-home\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:38.583606 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.583473    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c85d384e-b1c4-41a4-af96-acaf10b3b554" (UID: "c85d384e-b1c4-41a4-af96-acaf10b3b554"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:38.585403 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.585380    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-dshm" (OuterVolumeSpecName: "dshm") pod "c85d384e-b1c4-41a4-af96-acaf10b3b554" (UID: "c85d384e-b1c4-41a4-af96-acaf10b3b554"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:38.585683 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.585658    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c85d384e-b1c4-41a4-af96-acaf10b3b554-kube-api-access-9k9l4" (OuterVolumeSpecName: "kube-api-access-9k9l4") pod "c85d384e-b1c4-41a4-af96-acaf10b3b554" (UID: "c85d384e-b1c4-41a4-af96-acaf10b3b554"). InnerVolumeSpecName "kube-api-access-9k9l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:38.585683 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.585659    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c85d384e-b1c4-41a4-af96-acaf10b3b554-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c85d384e-b1c4-41a4-af96-acaf10b3b554" (UID: "c85d384e-b1c4-41a4-af96-acaf10b3b554"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:41:38.684112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.684074    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-model-cache\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:38.684112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.684104    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-dshm\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:38.684112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.684113    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c85d384e-b1c4-41a4-af96-acaf10b3b554-tmp-dir\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:38.684112 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.684122    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k9l4\" (UniqueName: \"kubernetes.io/projected/c85d384e-b1c4-41a4-af96-acaf10b3b554-kube-api-access-9k9l4\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:38.684373 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:38.684133    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c85d384e-b1c4-41a4-af96-acaf10b3b554-tls-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:39.182154 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.182118    2563 generic.go:356] "Generic (PLEG): container finished" podID="c85d384e-b1c4-41a4-af96-acaf10b3b554" containerID="112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4" exitCode=0
Jul 28 19:41:39.182319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.182166    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"
Jul 28 19:41:39.182319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.182186    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d" event={"ID":"c85d384e-b1c4-41a4-af96-acaf10b3b554","Type":"ContainerDied","Data":"112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4"}
Jul 28 19:41:39.182319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.182225    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d" event={"ID":"c85d384e-b1c4-41a4-af96-acaf10b3b554","Type":"ContainerDied","Data":"143cfde38933a85426b1572d9e9b42818525dabb6a305f93b7f9831a795453ff"}
Jul 28 19:41:39.182319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.182248    2563 scope.go:117] "RemoveContainer" containerID="112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4"
Jul 28 19:41:39.190616 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.190596    2563 scope.go:117] "RemoveContainer" containerID="112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4"
Jul 28 19:41:39.190868 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:41:39.190847    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4\": container with ID starting with 112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4 not found: ID does not exist" containerID="112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4"
Jul 28 19:41:39.190907 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.190884    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4"} err="failed to get container status \"112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4\": rpc error: code = NotFound desc = could not find container \"112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4\": container with ID starting with 112a30143f21f1954f40c4f9fb48ada122d31ac6002066d1d6098bf2f4948bd4 not found: ID does not exist"
Jul 28 19:41:39.197990 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.197965    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"]
Jul 28 19:41:39.201210 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.201190    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-5f6911/test-model-5f6911-kserve-76f59dbbcc-wbz7d"]
Jul 28 19:41:39.257389 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.257361    2563 scope.go:117] "RemoveContainer" containerID="45cf0d83ac258bb9ca5823a4af7009c609c0ba08da1636dcdb3c0e2c988e4cf0"
Jul 28 19:41:39.940648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:39.940609    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:41:40.494622 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:40.494586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:40.721480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:40.721435    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"]
Jul 28 19:41:40.721666 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:40.721649    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy" containerID="cri-o://e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4" gracePeriod=30
Jul 28 19:41:41.143095 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:41.143063    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c85d384e-b1c4-41a4-af96-acaf10b3b554" path="/var/lib/kubelet/pods/c85d384e-b1c4-41a4-af96-acaf10b3b554/volumes"
Jul 28 19:41:41.990635 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:41.990598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:45.761680 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:45.761649    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:45.825905 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:45.825819    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:45.978236 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:45.978211    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:41:45.988549 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:45.988447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:46.063271 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.063241    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:46.149235 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149203    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-podinfo\") pod \"fcb7286e-2899-4a02-86dc-baacbf931eb2\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") "
Jul 28 19:41:46.149235 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149250    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-token\") pod \"fcb7286e-2899-4a02-86dc-baacbf931eb2\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") "
Jul 28 19:41:46.149493 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149282    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwff5\" (UniqueName: \"kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-kube-api-access-kwff5\") pod \"fcb7286e-2899-4a02-86dc-baacbf931eb2\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") "
Jul 28 19:41:46.149493 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149391    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-credential-socket\") pod \"fcb7286e-2899-4a02-86dc-baacbf931eb2\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") "
Jul 28 19:41:46.149493 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149429    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb7286e-2899-4a02-86dc-baacbf931eb2-istiod-ca-cert\") pod \"fcb7286e-2899-4a02-86dc-baacbf931eb2\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") "
Jul 28 19:41:46.149493 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149477    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-envoy\") pod \"fcb7286e-2899-4a02-86dc-baacbf931eb2\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") "
Jul 28 19:41:46.149711 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149517    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-certs\") pod \"fcb7286e-2899-4a02-86dc-baacbf931eb2\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") "
Jul 28 19:41:46.149711 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149580    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-data\") pod \"fcb7286e-2899-4a02-86dc-baacbf931eb2\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") "
Jul 28 19:41:46.149711 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149609    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-socket\") pod \"fcb7286e-2899-4a02-86dc-baacbf931eb2\" (UID: \"fcb7286e-2899-4a02-86dc-baacbf931eb2\") "
Jul 28 19:41:46.149870 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149845    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcb7286e-2899-4a02-86dc-baacbf931eb2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fcb7286e-2899-4a02-86dc-baacbf931eb2" (UID: "fcb7286e-2899-4a02-86dc-baacbf931eb2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:41:46.149870 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149837    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fcb7286e-2899-4a02-86dc-baacbf931eb2" (UID: "fcb7286e-2899-4a02-86dc-baacbf931eb2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:46.150024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.149951    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fcb7286e-2899-4a02-86dc-baacbf931eb2" (UID: "fcb7286e-2899-4a02-86dc-baacbf931eb2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:46.150248 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.150087    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fcb7286e-2899-4a02-86dc-baacbf931eb2" (UID: "fcb7286e-2899-4a02-86dc-baacbf931eb2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:46.150511 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.150493    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-data" (OuterVolumeSpecName: "istio-data") pod "fcb7286e-2899-4a02-86dc-baacbf931eb2" (UID: "fcb7286e-2899-4a02-86dc-baacbf931eb2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:46.151763 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.151738    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-kube-api-access-kwff5" (OuterVolumeSpecName: "kube-api-access-kwff5") pod "fcb7286e-2899-4a02-86dc-baacbf931eb2" (UID: "fcb7286e-2899-4a02-86dc-baacbf931eb2"). InnerVolumeSpecName "kube-api-access-kwff5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:46.151862 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.151761    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fcb7286e-2899-4a02-86dc-baacbf931eb2" (UID: "fcb7286e-2899-4a02-86dc-baacbf931eb2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:41:46.151862 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.151778    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-token" (OuterVolumeSpecName: "istio-token") pod "fcb7286e-2899-4a02-86dc-baacbf931eb2" (UID: "fcb7286e-2899-4a02-86dc-baacbf931eb2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:46.151862 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.151837    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fcb7286e-2899-4a02-86dc-baacbf931eb2" (UID: "fcb7286e-2899-4a02-86dc-baacbf931eb2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:46.212220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.212180    2563 generic.go:356] "Generic (PLEG): container finished" podID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerID="e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4" exitCode=0
Jul 28 19:41:46.212365 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.212247    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"
Jul 28 19:41:46.212365 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.212267    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" event={"ID":"fcb7286e-2899-4a02-86dc-baacbf931eb2","Type":"ContainerDied","Data":"e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4"}
Jul 28 19:41:46.212365 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.212329    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw" event={"ID":"fcb7286e-2899-4a02-86dc-baacbf931eb2","Type":"ContainerDied","Data":"c51226684cd8dfa8d2877297d9940cb2cf896191778602ced1dbf59e656ea32f"}
Jul 28 19:41:46.212365 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.212357    2563 scope.go:117] "RemoveContainer" containerID="e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4"
Jul 28 19:41:46.222133 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.222112    2563 scope.go:117] "RemoveContainer" containerID="2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8"
Jul 28 19:41:46.229781 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.229762    2563 scope.go:117] "RemoveContainer" containerID="e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4"
Jul 28 19:41:46.230057 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:41:46.230036    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4\": container with ID starting with e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4 not found: ID does not exist" containerID="e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4"
Jul 28 19:41:46.230133 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.230070    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4"} err="failed to get container status \"e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4\": rpc error: code = NotFound desc = could not find container \"e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4\": container with ID starting with e03c765a8a3ef83a1336a498d13aa27f8dfd67da460212c3d62aa13b36e248e4 not found: ID does not exist"
Jul 28 19:41:46.230133 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.230094    2563 scope.go:117] "RemoveContainer" containerID="2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8"
Jul 28 19:41:46.230344 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:41:46.230325    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8\": container with ID starting with 2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8 not found: ID does not exist" containerID="2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8"
Jul 28 19:41:46.230385 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.230351    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8"} err="failed to get container status \"2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8\": rpc error: code = NotFound desc = could not find container \"2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8\": container with ID starting with 2db4bc1edd4f7b34f96308fc6dcf34279c89691cc818e12b9ef75686f87aaee8 not found: ID does not exist"
Jul 28 19:41:46.234450 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.234429    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"]
Jul 28 19:41:46.236855 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.236835    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e55e7d-openshift-default-6799fcc5b8-9ctvw"]
Jul 28 19:41:46.250622 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.250568    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.250622 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.250588    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb7286e-2899-4a02-86dc-baacbf931eb2-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.250622 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.250598    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.250622 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.250606    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.250622 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.250613    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.250622 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.250622    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb7286e-2899-4a02-86dc-baacbf931eb2-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.250852 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.250630    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.250852 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.250638    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.250852 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.250645    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwff5\" (UniqueName: \"kubernetes.io/projected/fcb7286e-2899-4a02-86dc-baacbf931eb2-kube-api-access-kwff5\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.255718 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:46.255691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:47.143401 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:47.143369    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" path="/var/lib/kubelet/pods/fcb7286e-2899-4a02-86dc-baacbf931eb2/volumes"
Jul 28 19:41:48.038436 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:48.038400    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:41:51.992208 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:51.992172    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:52.098769 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:52.098733    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:52.237436 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:52.237404    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"]
Jul 28 19:41:52.237637 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:52.237614    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy" containerID="cri-o://af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1" gracePeriod=30
Jul 28 19:41:52.260646 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:52.260535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:52.484099 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:52.484062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:52.847670 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:52.847636    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:53.061584 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.061553    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"]
Jul 28 19:41:53.061960 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.061896    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c85d384e-b1c4-41a4-af96-acaf10b3b554" containerName="main"
Jul 28 19:41:53.061960 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.061907    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c85d384e-b1c4-41a4-af96-acaf10b3b554" containerName="main"
Jul 28 19:41:53.061960 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.061935    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy"
Jul 28 19:41:53.061960 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.061941    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy"
Jul 28 19:41:53.061960 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.061947    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy"
Jul 28 19:41:53.061960 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.061953    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy"
Jul 28 19:41:53.062173 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.062013    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy"
Jul 28 19:41:53.062173 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.062021    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcb7286e-2899-4a02-86dc-baacbf931eb2" containerName="istio-proxy"
Jul 28 19:41:53.062173 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.062028    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c85d384e-b1c4-41a4-af96-acaf10b3b554" containerName="main"
Jul 28 19:41:53.066688 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.066671    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.069235 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.069211    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-29254e-openshift-default-dockercfg-lhc4d\""
Jul 28 19:41:53.076450 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.076430    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"]
Jul 28 19:41:53.083174 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.083148    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:53.212361 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.212276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-envoy\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.212361 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.212325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5l7z\" (UniqueName: \"kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-kube-api-access-b5l7z\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.212579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.212377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-data\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.212579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.212418    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-podinfo\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.212579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.212440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istiod-ca-cert\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.212579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.212464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-certs\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.212579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.212517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-socket\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.212579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.212537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-credential-socket\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.212579 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.212562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-token\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.253828 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.253790    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:53.312994 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.312949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-certs\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313179 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-socket\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313179 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-credential-socket\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313179 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-token\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313322 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-envoy\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313371 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313336    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-socket\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313422 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5l7z\" (UniqueName: \"kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-kube-api-access-b5l7z\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313422 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-certs\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-data\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-credential-socket\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-podinfo\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313649 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istiod-ca-cert\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.313763 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.313735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-data\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.314140 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.314124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istiod-ca-cert\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.315777 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.315756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-envoy\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.315945 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.315907    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-podinfo\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.322569 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.322545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-token\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.322771 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.322752    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5l7z\" (UniqueName: \"kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-kube-api-access-b5l7z\") pod \"e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") " pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.378071 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.378029    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:53.518148 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.518119    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"]
Jul 28 19:41:53.520884 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:41:53.520859    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f7928d2_eb01_4a7c_bd22_1aad11a029b6.slice/crio-33ed9925cb5f80ae00018ad27bef244eda634ae09cfa7d281171a2f308a180b4 WatchSource:0}: Error finding container 33ed9925cb5f80ae00018ad27bef244eda634ae09cfa7d281171a2f308a180b4: Status 404 returned error can't find the container with id 33ed9925cb5f80ae00018ad27bef244eda634ae09cfa7d281171a2f308a180b4
Jul 28 19:41:53.523361 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.523321    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:41:53.523449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.523394    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:41:53.523449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:53.523420    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:41:54.248189 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:54.248156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" event={"ID":"0f7928d2-eb01-4a7c-bd22-1aad11a029b6","Type":"ContainerStarted","Data":"dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9"}
Jul 28 19:41:54.248596 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:54.248229    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" event={"ID":"0f7928d2-eb01-4a7c-bd22-1aad11a029b6","Type":"ContainerStarted","Data":"33ed9925cb5f80ae00018ad27bef244eda634ae09cfa7d281171a2f308a180b4"}
Jul 28 19:41:54.267632 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:54.267584    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" podStartSLOduration=1.267566864 podStartE2EDuration="1.267566864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:54.266298621 +0000 UTC m=+2355.727906325" watchObservedRunningTime="2026-07-28 19:41:54.267566864 +0000 UTC m=+2355.729174566"
Jul 28 19:41:54.378953 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:54.378910    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:54.385281 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:54.385254    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:55.252310 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:55.252282    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:55.253266 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:55.253247    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:41:56.144473 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:56.144438    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:56.240461 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:56.240429    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:56.302907 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:56.302872    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:56.694169 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:56.694134    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:56.812291 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:56.812263    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:56.912803 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:56.912772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:57.495007 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.494984    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:41:57.553250 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553173    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-certs\") pod \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") "
Jul 28 19:41:57.553250 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553216    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-socket\") pod \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") "
Jul 28 19:41:57.553250 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553242    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-podinfo\") pod \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") "
Jul 28 19:41:57.553485 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553267    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-token\") pod \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") "
Jul 28 19:41:57.553485 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553303    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85rpw\" (UniqueName: \"kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-kube-api-access-85rpw\") pod \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") "
Jul 28 19:41:57.553485 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553338    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-credential-socket\") pod \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") "
Jul 28 19:41:57.553485 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553361    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-data\") pod \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") "
Jul 28 19:41:57.553485 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553392    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istiod-ca-cert\") pod \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") "
Jul 28 19:41:57.553485 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553433    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-envoy\") pod \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\" (UID: \"ed4ba6b5-0f98-4501-8e9e-3790984e243f\") "
Jul 28 19:41:57.553709 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553494    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed4ba6b5-0f98-4501-8e9e-3790984e243f" (UID: "ed4ba6b5-0f98-4501-8e9e-3790984e243f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:57.553709 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553519    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed4ba6b5-0f98-4501-8e9e-3790984e243f" (UID: "ed4ba6b5-0f98-4501-8e9e-3790984e243f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:57.553815 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553759    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:57.553815 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553779    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:57.553815 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553792    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed4ba6b5-0f98-4501-8e9e-3790984e243f" (UID: "ed4ba6b5-0f98-4501-8e9e-3790984e243f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:57.553955 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.553822    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed4ba6b5-0f98-4501-8e9e-3790984e243f" (UID: "ed4ba6b5-0f98-4501-8e9e-3790984e243f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:41:57.554522 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.554496    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed4ba6b5-0f98-4501-8e9e-3790984e243f" (UID: "ed4ba6b5-0f98-4501-8e9e-3790984e243f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:57.555712 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.555691    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed4ba6b5-0f98-4501-8e9e-3790984e243f" (UID: "ed4ba6b5-0f98-4501-8e9e-3790984e243f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:41:57.555848 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.555718    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed4ba6b5-0f98-4501-8e9e-3790984e243f" (UID: "ed4ba6b5-0f98-4501-8e9e-3790984e243f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:57.555848 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.555771    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed4ba6b5-0f98-4501-8e9e-3790984e243f" (UID: "ed4ba6b5-0f98-4501-8e9e-3790984e243f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:57.556394 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.556357    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-kube-api-access-85rpw" (OuterVolumeSpecName: "kube-api-access-85rpw") pod "ed4ba6b5-0f98-4501-8e9e-3790984e243f" (UID: "ed4ba6b5-0f98-4501-8e9e-3790984e243f"). InnerVolumeSpecName "kube-api-access-85rpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:57.654163 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.654133    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:57.654163 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.654159    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:57.654163 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.654169    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:57.654386 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.654177    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85rpw\" (UniqueName: \"kubernetes.io/projected/ed4ba6b5-0f98-4501-8e9e-3790984e243f-kube-api-access-85rpw\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:57.654386 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.654187    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:57.654386 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.654195    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:57.654386 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:57.654203    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4ba6b5-0f98-4501-8e9e-3790984e243f-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:41:58.267179 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.267145    2563 generic.go:356] "Generic (PLEG): container finished" podID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerID="af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1" exitCode=0
Jul 28 19:41:58.267179 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.267183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" event={"ID":"ed4ba6b5-0f98-4501-8e9e-3790984e243f","Type":"ContainerDied","Data":"af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1"}
Jul 28 19:41:58.267427 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.267197    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"
Jul 28 19:41:58.267427 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.267223    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf" event={"ID":"ed4ba6b5-0f98-4501-8e9e-3790984e243f","Type":"ContainerDied","Data":"ef758079e623dd237732c34665d4f34b9ebad6d47a110d20593db90e3d4c9faa"}
Jul 28 19:41:58.267427 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.267244    2563 scope.go:117] "RemoveContainer" containerID="af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1"
Jul 28 19:41:58.276355 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.276339    2563 scope.go:117] "RemoveContainer" containerID="873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e"
Jul 28 19:41:58.288804 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.288779    2563 scope.go:117] "RemoveContainer" containerID="af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1"
Jul 28 19:41:58.289164 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:41:58.289141    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1\": container with ID starting with af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1 not found: ID does not exist" containerID="af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1"
Jul 28 19:41:58.289268 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.289175    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1"} err="failed to get container status \"af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1\": rpc error: code = NotFound desc = could not find container \"af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1\": container with ID starting with af76f2eae2dfedb5fa9a38b84729cae7ae8a99eff3b8df75bd30c0472a9ab7e1 not found: ID does not exist"
Jul 28 19:41:58.289268 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.289199    2563 scope.go:117] "RemoveContainer" containerID="873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e"
Jul 28 19:41:58.289268 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.289220    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"]
Jul 28 19:41:58.289489 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:41:58.289472    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e\": container with ID starting with 873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e not found: ID does not exist" containerID="873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e"
Jul 28 19:41:58.289528 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.289497    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e"} err="failed to get container status \"873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e\": rpc error: code = NotFound desc = could not find container \"873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e\": container with ID starting with 873e7a4452d62d7d77f75ba92a855165418e72a8aa523d0e00ed2360669add9e not found: ID does not exist"
Jul 28 19:41:58.293392 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:58.293372    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5f6911-openshift-default-5796b446bd-nnvbf"]
Jul 28 19:41:59.143878 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:41:59.143845    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" path="/var/lib/kubelet/pods/ed4ba6b5-0f98-4501-8e9e-3790984e243f/volumes"
Jul 28 19:42:00.937094 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:00.937055    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:42:09.320338 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:09.320301    2563 generic.go:356] "Generic (PLEG): container finished" podID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerID="dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9" exitCode=0
Jul 28 19:42:09.320762 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:09.320377    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" event={"ID":"0f7928d2-eb01-4a7c-bd22-1aad11a029b6","Type":"ContainerDied","Data":"dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9"}
Jul 28 19:42:09.320804 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:09.320761    2563 scope.go:117] "RemoveContainer" containerID="dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9"
Jul 28 19:42:09.321700 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:09.321674    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:42:09.321756 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:09.321743    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:42:09.321809 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:09.321798    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:42:10.252345 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:10.252305    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:42:10.327555 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:10.327517    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" event={"ID":"0f7928d2-eb01-4a7c-bd22-1aad11a029b6","Type":"ContainerStarted","Data":"06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22"}
Jul 28 19:42:10.378525 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:10.378493    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:42:10.379529 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:10.379504    2563 patch_prober.go:28] interesting pod/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 28 19:42:10.379697 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:10.379544    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 28 19:42:11.378589 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:11.378551    2563 patch_prober.go:28] interesting pod/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 28 19:42:11.379002 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:11.378620    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 28 19:42:12.378586 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:12.378552    2563 patch_prober.go:28] interesting pod/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 28 19:42:12.378768 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:12.378608    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 28 19:42:13.378454 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:13.378410    2563 patch_prober.go:28] interesting pod/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 28 19:42:13.378646 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:13.378476    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 28 19:42:14.382820 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:14.382781    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:42:14.383201 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:14.383110    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:42:14.383688 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:14.383669    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:42:21.101473 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.101423    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:21.553411 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.553314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:21.819014 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.818983    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"]
Jul 28 19:42:21.819347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.819321    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy"
Jul 28 19:42:21.819347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.819345    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy"
Jul 28 19:42:21.819487 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.819370    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy"
Jul 28 19:42:21.819487 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.819380    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy"
Jul 28 19:42:21.819487 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.819471    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy"
Jul 28 19:42:21.822399 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.822380    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:21.825076 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.825057    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-7a6162-openshift-default-dockercfg-44pdr\""
Jul 28 19:42:21.834991 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.834965    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"]
Jul 28 19:42:21.849300 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.849274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:21.983851 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.983815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-certs\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:21.984047 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.983863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-istio-token\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:21.984047 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.983884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d920e673-0200-4073-aaf0-026b1bf1f585-istiod-ca-cert\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:21.984047 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.983962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d920e673-0200-4073-aaf0-026b1bf1f585-istio-podinfo\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:21.984047 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.983986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-credential-socket\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:21.984047 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.984006    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-envoy\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:21.984047 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.984029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-socket\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:21.984251 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.984092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4995k\" (UniqueName: \"kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-kube-api-access-4995k\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:21.984251 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:21.984123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-data\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.084580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.084487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4995k\" (UniqueName: \"kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-kube-api-access-4995k\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.084580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.084534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-data\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.084580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.084577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-certs\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.084890 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.084614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-istio-token\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.084890 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.084636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d920e673-0200-4073-aaf0-026b1bf1f585-istiod-ca-cert\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.084890 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.084665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d920e673-0200-4073-aaf0-026b1bf1f585-istio-podinfo\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.084890 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.084694    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-credential-socket\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.084890 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.084867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-envoy\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.085143 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.084912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-socket\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.085143 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.085053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-data\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.085143 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.085096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-certs\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.085294 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.085190    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-socket\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.085347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.085301    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-credential-socket\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.085409 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.085394    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d920e673-0200-4073-aaf0-026b1bf1f585-istiod-ca-cert\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.087131 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.087113    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-envoy\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.087364 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.087344    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d920e673-0200-4073-aaf0-026b1bf1f585-istio-podinfo\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.092461 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.092435    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-istio-token\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.093151 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.093129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4995k\" (UniqueName: \"kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-kube-api-access-4995k\") pod \"e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") " pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.132747 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.132716    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:22.170735 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.170708    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:22.276180 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.276149    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"]
Jul 28 19:42:22.278880 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:42:22.278852    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd920e673_0200_4073_aaf0_026b1bf1f585.slice/crio-4e074354776c64a30f2c05cb9924dd3991002ae6bcd140f1adda459f12c96181 WatchSource:0}: Error finding container 4e074354776c64a30f2c05cb9924dd3991002ae6bcd140f1adda459f12c96181: Status 404 returned error can't find the container with id 4e074354776c64a30f2c05cb9924dd3991002ae6bcd140f1adda459f12c96181
Jul 28 19:42:22.281183 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.281152    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:42:22.281287 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.281216    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:42:22.281287 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.281250    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:42:22.385314 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.385212    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" event={"ID":"d920e673-0200-4073-aaf0-026b1bf1f585","Type":"ContainerStarted","Data":"708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa"}
Jul 28 19:42:22.385314 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.385273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" event={"ID":"d920e673-0200-4073-aaf0-026b1bf1f585","Type":"ContainerStarted","Data":"4e074354776c64a30f2c05cb9924dd3991002ae6bcd140f1adda459f12c96181"}
Jul 28 19:42:22.405248 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:22.405163    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" podStartSLOduration=1.405144866 podStartE2EDuration="1.405144866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:22.403158722 +0000 UTC m=+2383.864766449" watchObservedRunningTime="2026-07-28 19:42:22.405144866 +0000 UTC m=+2383.866752568"
Jul 28 19:42:23.133230 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:23.133199    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:23.143379 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:23.143352    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:23.389969 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:23.389865    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:23.390906 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:23.390887    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:24.327599 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:24.327568    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:24.401582 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:24.401540    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:24.466689 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:24.466654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:24.812285 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:24.812193    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:25.039288 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:25.039248    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:25.141207 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:25.141174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:31.645700 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:31.645660    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:42:39.113677 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.113646    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:42:39.117068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.117049    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:42:39.167135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.167087    2563 patch_prober.go:28] interesting pod/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:56318->10.134.0.45:15021: read: connection reset by peer" start-of-body=
Jul 28 19:42:39.167259 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.167165    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:56318->10.134.0.45:15021: read: connection reset by peer"
Jul 28 19:42:39.279799 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.279768    2563 scope.go:117] "RemoveContainer" containerID="f8ce8e6e8b08e747a4d5088988a4859caa6b53591ac721c0320f4b64e27533e9"
Jul 28 19:42:39.466501 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.466398    2563 generic.go:356] "Generic (PLEG): container finished" podID="d920e673-0200-4073-aaf0-026b1bf1f585" containerID="708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa" exitCode=0
Jul 28 19:42:39.466501 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.466470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" event={"ID":"d920e673-0200-4073-aaf0-026b1bf1f585","Type":"ContainerDied","Data":"708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa"}
Jul 28 19:42:39.466913 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.466896    2563 scope.go:117] "RemoveContainer" containerID="708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa"
Jul 28 19:42:39.467936 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.467894    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:42:39.468028 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.467982    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:42:39.468070 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:39.468025    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:42:40.472662 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:40.472630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" event={"ID":"d920e673-0200-4073-aaf0-026b1bf1f585","Type":"ContainerStarted","Data":"4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2"}
Jul 28 19:42:41.133103 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:41.133066    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:41.133306 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:41.133283    2563 patch_prober.go:28] interesting pod/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 28 19:42:41.133352 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:41.133333    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 28 19:42:42.133365 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:42.133327    2563 patch_prober.go:28] interesting pod/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 28 19:42:42.133746 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:42.133388    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 28 19:42:43.133254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:43.133207    2563 patch_prober.go:28] interesting pod/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 28 19:42:43.133472 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:43.133273    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 28 19:42:44.133712 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:44.133668    2563 patch_prober.go:28] interesting pod/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 28 19:42:44.134171 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:44.133726    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 28 19:42:45.137121 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:45.137094    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:45.137552 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:45.137440    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:45.142777 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:45.142755    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:42:49.325213 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:42:49.325174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:07.914383 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:07.914284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:08.011795 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:08.011762    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:08.093938 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:08.093885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:08.460066 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:08.460033    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:10.535593 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:10.535552    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:43:14.215049 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:14.215007    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:14.317538 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:14.317488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:14.536050 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:14.535958    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"]
Jul 28 19:43:14.536217 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:14.536160    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy" containerID="cri-o://06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22" gracePeriod=30
Jul 28 19:43:14.552854 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:14.552830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:14.759275 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:14.759237    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:19.776421 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.776398    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:43:19.904078 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904043    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-credential-socket\") pod \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") "
Jul 28 19:43:19.904270 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904088    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5l7z\" (UniqueName: \"kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-kube-api-access-b5l7z\") pod \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") "
Jul 28 19:43:19.904270 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904130    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-token\") pod \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") "
Jul 28 19:43:19.904270 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904154    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-certs\") pod \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") "
Jul 28 19:43:19.904270 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904243    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-podinfo\") pod \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") "
Jul 28 19:43:19.904270 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904266    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-socket\") pod \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") "
Jul 28 19:43:19.904539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904323    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-envoy\") pod \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") "
Jul 28 19:43:19.904539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904347    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-data\") pod \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") "
Jul 28 19:43:19.904539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904372    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istiod-ca-cert\") pod \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\" (UID: \"0f7928d2-eb01-4a7c-bd22-1aad11a029b6\") "
Jul 28 19:43:19.904539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904371    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0f7928d2-eb01-4a7c-bd22-1aad11a029b6" (UID: "0f7928d2-eb01-4a7c-bd22-1aad11a029b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:19.904741 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904571    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0f7928d2-eb01-4a7c-bd22-1aad11a029b6" (UID: "0f7928d2-eb01-4a7c-bd22-1aad11a029b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:19.904741 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904620    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:19.904859 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.904817    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0f7928d2-eb01-4a7c-bd22-1aad11a029b6" (UID: "0f7928d2-eb01-4a7c-bd22-1aad11a029b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:19.905165 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.905106    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0f7928d2-eb01-4a7c-bd22-1aad11a029b6" (UID: "0f7928d2-eb01-4a7c-bd22-1aad11a029b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:43:19.905165 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.905121    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-data" (OuterVolumeSpecName: "istio-data") pod "0f7928d2-eb01-4a7c-bd22-1aad11a029b6" (UID: "0f7928d2-eb01-4a7c-bd22-1aad11a029b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:19.906456 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.906430    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-kube-api-access-b5l7z" (OuterVolumeSpecName: "kube-api-access-b5l7z") pod "0f7928d2-eb01-4a7c-bd22-1aad11a029b6" (UID: "0f7928d2-eb01-4a7c-bd22-1aad11a029b6"). InnerVolumeSpecName "kube-api-access-b5l7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:19.906701 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.906682    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0f7928d2-eb01-4a7c-bd22-1aad11a029b6" (UID: "0f7928d2-eb01-4a7c-bd22-1aad11a029b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:19.906955 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.906911    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0f7928d2-eb01-4a7c-bd22-1aad11a029b6" (UID: "0f7928d2-eb01-4a7c-bd22-1aad11a029b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:43:19.907049 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:19.907020    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f7928d2-eb01-4a7c-bd22-1aad11a029b6" (UID: "0f7928d2-eb01-4a7c-bd22-1aad11a029b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:20.005544 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.005507    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:20.005544 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.005540    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:20.005544 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.005550    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:20.005792 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.005582    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:20.005792 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.005591    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:20.005792 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.005599    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5l7z\" (UniqueName: \"kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-kube-api-access-b5l7z\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:20.005792 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.005607    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:20.005792 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.005616    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f7928d2-eb01-4a7c-bd22-1aad11a029b6-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:20.646464 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.646426    2563 generic.go:356] "Generic (PLEG): container finished" podID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerID="06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22" exitCode=0
Jul 28 19:43:20.646648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.646476    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"
Jul 28 19:43:20.646648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.646483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" event={"ID":"0f7928d2-eb01-4a7c-bd22-1aad11a029b6","Type":"ContainerDied","Data":"06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22"}
Jul 28 19:43:20.646648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.646533    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454" event={"ID":"0f7928d2-eb01-4a7c-bd22-1aad11a029b6","Type":"ContainerDied","Data":"33ed9925cb5f80ae00018ad27bef244eda634ae09cfa7d281171a2f308a180b4"}
Jul 28 19:43:20.646648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.646559    2563 scope.go:117] "RemoveContainer" containerID="06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22"
Jul 28 19:43:20.657147 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.657114    2563 scope.go:117] "RemoveContainer" containerID="dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9"
Jul 28 19:43:20.665739 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.665722    2563 scope.go:117] "RemoveContainer" containerID="06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22"
Jul 28 19:43:20.666033 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:43:20.666010    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22\": container with ID starting with 06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22 not found: ID does not exist" containerID="06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22"
Jul 28 19:43:20.666124 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.666041    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22"} err="failed to get container status \"06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22\": rpc error: code = NotFound desc = could not find container \"06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22\": container with ID starting with 06f4a5eb715299ae85a3599961a0f47cd5ae7aacc8c25d5e5a6ddbaedf9dab22 not found: ID does not exist"
Jul 28 19:43:20.666124 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.666059    2563 scope.go:117] "RemoveContainer" containerID="dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9"
Jul 28 19:43:20.666321 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:43:20.666306    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9\": container with ID starting with dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9 not found: ID does not exist" containerID="dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9"
Jul 28 19:43:20.666369 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.666327    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9"} err="failed to get container status \"dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9\": rpc error: code = NotFound desc = could not find container \"dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9\": container with ID starting with dd019b4a2cb9adf1d10162c4839cb16f1f4fbd254047bbdf1bed4d4a0ba1baf9 not found: ID does not exist"
Jul 28 19:43:20.668861 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.668834    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"]
Jul 28 19:43:20.672288 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:20.672265    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-29254e-openshift-default-6f947fdbc4-jp454"]
Jul 28 19:43:21.143013 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:21.142986    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" path="/var/lib/kubelet/pods/0f7928d2-eb01-4a7c-bd22-1aad11a029b6/volumes"
Jul 28 19:43:30.412323 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:30.412292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:30.479030 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:30.478981    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:30.532900 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:30.532868    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:30.585214 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:30.585184    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:30.950347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:30.950316    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:35.143994 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:35.143962    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:43:36.838810 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:36.838771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:36.897159 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:36.897116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:37.086086 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.086050    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"]
Jul 28 19:43:37.086291 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.086266    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy" containerID="cri-o://4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2" gracePeriod=30
Jul 28 19:43:37.118060 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.117971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:37.179436 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.179405    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:37.360420 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.360382    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:37.666694 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.666658    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:37.940083 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.939999    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"]
Jul 28 19:43:37.940445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.940303    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy"
Jul 28 19:43:37.940445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.940314    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy"
Jul 28 19:43:37.940445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.940324    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy"
Jul 28 19:43:37.940445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.940329    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy"
Jul 28 19:43:37.940445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.940387    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy"
Jul 28 19:43:37.940445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.940395    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed4ba6b5-0f98-4501-8e9e-3790984e243f" containerName="istio-proxy"
Jul 28 19:43:37.944804 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.944784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:37.947886 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.947866    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-ec81e7-openshift-default-dockercfg-jxmkw\""
Jul 28 19:43:37.956488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.956465    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"]
Jul 28 19:43:37.960475 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:37.960444    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:38.049201 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.049158    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-token\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.049402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.049222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-envoy\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.049402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.049274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istiod-ca-cert\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.049402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.049297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-credential-socket\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.049402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.049322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-data\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.049402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.049350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mf5f\" (UniqueName: \"kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-kube-api-access-4mf5f\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.049402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.049392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-socket\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.049700 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.049417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-podinfo\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.049700 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.049446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-certs\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.150678 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.150642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-socket\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.150678 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.150679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-podinfo\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.150948 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.150698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-certs\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.150948 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.150731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-token\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.150948 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.150765    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-envoy\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.150948 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.150797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istiod-ca-cert\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.150948 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.150815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-credential-socket\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.150948 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.150840    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-data\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.150948 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.150913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mf5f\" (UniqueName: \"kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-kube-api-access-4mf5f\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.151383 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.151150    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-socket\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.151383 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.151183    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-credential-socket\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.151512 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.151383    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-data\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.151512 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.151428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-certs\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.151647 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.151623    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istiod-ca-cert\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.153299 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.153283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-envoy\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.153362 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.153342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-podinfo\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.159865 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.159843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-token\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.160645 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.160622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mf5f\" (UniqueName: \"kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-kube-api-access-4mf5f\") pod \"e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") " pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.255179 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.255096    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:38.260507 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.260476    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:38.395294 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.395238    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"]
Jul 28 19:43:38.397701 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:43:38.397672    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a57e1bf_5b68_4bc0_813f_b33d3ed09c7d.slice/crio-bc3783be19ddd19e1777564e765e5dedd0b90340431e7f2f2eeb1922e241834b WatchSource:0}: Error finding container bc3783be19ddd19e1777564e765e5dedd0b90340431e7f2f2eeb1922e241834b: Status 404 returned error can't find the container with id bc3783be19ddd19e1777564e765e5dedd0b90340431e7f2f2eeb1922e241834b
Jul 28 19:43:38.399751 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.399723    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:43:38.399828 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.399787    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:43:38.399828 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.399815    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:43:38.722640 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.722603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" event={"ID":"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d","Type":"ContainerStarted","Data":"25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25"}
Jul 28 19:43:38.722837 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.722655    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" event={"ID":"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d","Type":"ContainerStarted","Data":"bc3783be19ddd19e1777564e765e5dedd0b90340431e7f2f2eeb1922e241834b"}
Jul 28 19:43:38.741998 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:38.741940    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" podStartSLOduration=1.741902188 podStartE2EDuration="1.741902188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:43:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:43:38.739878069 +0000 UTC m=+2460.201485772" watchObservedRunningTime="2026-07-28 19:43:38.741902188 +0000 UTC m=+2460.203509891"
Jul 28 19:43:39.255797 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:39.255755    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:39.260566 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:39.260538    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:39.726775 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:39.726733    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:39.727885 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:39.727864    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:40.619599 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:40.619563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:40.718417 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:40.718383    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:40.818224 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:40.818183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:41.130752 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:41.130713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:41.400129 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:41.400036    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:41.512391 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:41.512352    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:42.340268 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.340242    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:43:42.387779 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.387684    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-socket\") pod \"d920e673-0200-4073-aaf0-026b1bf1f585\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") "
Jul 28 19:43:42.388087 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388064    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-credential-socket\") pod \"d920e673-0200-4073-aaf0-026b1bf1f585\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") "
Jul 28 19:43:42.388248 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388231    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d920e673-0200-4073-aaf0-026b1bf1f585-istiod-ca-cert\") pod \"d920e673-0200-4073-aaf0-026b1bf1f585\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") "
Jul 28 19:43:42.388462 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388404    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4995k\" (UniqueName: \"kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-kube-api-access-4995k\") pod \"d920e673-0200-4073-aaf0-026b1bf1f585\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") "
Jul 28 19:43:42.388462 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388149    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d920e673-0200-4073-aaf0-026b1bf1f585" (UID: "d920e673-0200-4073-aaf0-026b1bf1f585"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:42.388462 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d920e673-0200-4073-aaf0-026b1bf1f585" (UID: "d920e673-0200-4073-aaf0-026b1bf1f585"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:42.388685 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388488    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-data\") pod \"d920e673-0200-4073-aaf0-026b1bf1f585\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") "
Jul 28 19:43:42.388685 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388526    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d920e673-0200-4073-aaf0-026b1bf1f585-istio-podinfo\") pod \"d920e673-0200-4073-aaf0-026b1bf1f585\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") "
Jul 28 19:43:42.388685 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388566    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d920e673-0200-4073-aaf0-026b1bf1f585-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d920e673-0200-4073-aaf0-026b1bf1f585" (UID: "d920e673-0200-4073-aaf0-026b1bf1f585"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:43:42.388685 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388572    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-certs\") pod \"d920e673-0200-4073-aaf0-026b1bf1f585\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") "
Jul 28 19:43:42.388685 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388623    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-envoy\") pod \"d920e673-0200-4073-aaf0-026b1bf1f585\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") "
Jul 28 19:43:42.388975 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388752    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d920e673-0200-4073-aaf0-026b1bf1f585" (UID: "d920e673-0200-4073-aaf0-026b1bf1f585"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:42.388975 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.388805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-istio-token\") pod \"d920e673-0200-4073-aaf0-026b1bf1f585\" (UID: \"d920e673-0200-4073-aaf0-026b1bf1f585\") "
Jul 28 19:43:42.389113 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.389093    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:42.389169 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.389120    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:42.389169 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.389134    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:42.389169 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.389147    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d920e673-0200-4073-aaf0-026b1bf1f585-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:42.389902 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.389796    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-data" (OuterVolumeSpecName: "istio-data") pod "d920e673-0200-4073-aaf0-026b1bf1f585" (UID: "d920e673-0200-4073-aaf0-026b1bf1f585"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:42.391311 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.391273    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-kube-api-access-4995k" (OuterVolumeSpecName: "kube-api-access-4995k") pod "d920e673-0200-4073-aaf0-026b1bf1f585" (UID: "d920e673-0200-4073-aaf0-026b1bf1f585"). InnerVolumeSpecName "kube-api-access-4995k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:42.391732 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.391710    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d920e673-0200-4073-aaf0-026b1bf1f585-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d920e673-0200-4073-aaf0-026b1bf1f585" (UID: "d920e673-0200-4073-aaf0-026b1bf1f585"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:43:42.392192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.392169    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-istio-token" (OuterVolumeSpecName: "istio-token") pod "d920e673-0200-4073-aaf0-026b1bf1f585" (UID: "d920e673-0200-4073-aaf0-026b1bf1f585"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:42.392348 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.392329    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d920e673-0200-4073-aaf0-026b1bf1f585" (UID: "d920e673-0200-4073-aaf0-026b1bf1f585"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:43:42.490432 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.490390    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d920e673-0200-4073-aaf0-026b1bf1f585-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:42.490432 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.490424    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:42.490432 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.490432    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:42.490432 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.490440    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4995k\" (UniqueName: \"kubernetes.io/projected/d920e673-0200-4073-aaf0-026b1bf1f585-kube-api-access-4995k\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:42.490695 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.490449    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d920e673-0200-4073-aaf0-026b1bf1f585-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:43:42.741301 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.741208    2563 generic.go:356] "Generic (PLEG): container finished" podID="d920e673-0200-4073-aaf0-026b1bf1f585" containerID="4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2" exitCode=0
Jul 28 19:43:42.741301 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.741253    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"
Jul 28 19:43:42.741301 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.741279    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" event={"ID":"d920e673-0200-4073-aaf0-026b1bf1f585","Type":"ContainerDied","Data":"4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2"}
Jul 28 19:43:42.741525 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.741323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2" event={"ID":"d920e673-0200-4073-aaf0-026b1bf1f585","Type":"ContainerDied","Data":"4e074354776c64a30f2c05cb9924dd3991002ae6bcd140f1adda459f12c96181"}
Jul 28 19:43:42.741525 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.741344    2563 scope.go:117] "RemoveContainer" containerID="4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2"
Jul 28 19:43:42.750882 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.750864    2563 scope.go:117] "RemoveContainer" containerID="708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa"
Jul 28 19:43:42.759224 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.759204    2563 scope.go:117] "RemoveContainer" containerID="4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2"
Jul 28 19:43:42.759501 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:43:42.759481    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2\": container with ID starting with 4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2 not found: ID does not exist" containerID="4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2"
Jul 28 19:43:42.759575 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.759515    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2"} err="failed to get container status \"4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2\": rpc error: code = NotFound desc = could not find container \"4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2\": container with ID starting with 4f9c8c0eee811b1ca5a2f0361df1f90073004df52777dc20ee2a4d09d48fcce2 not found: ID does not exist"
Jul 28 19:43:42.759575 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.759542    2563 scope.go:117] "RemoveContainer" containerID="708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa"
Jul 28 19:43:42.759791 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:43:42.759773    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa\": container with ID starting with 708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa not found: ID does not exist" containerID="708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa"
Jul 28 19:43:42.759835 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.759798    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa"} err="failed to get container status \"708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa\": rpc error: code = NotFound desc = could not find container \"708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa\": container with ID starting with 708fc23c617f0b0858a847b25d6b7c593eb7a82995db10d6a83dc2138c614ffa not found: ID does not exist"
Jul 28 19:43:42.764848 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.764824    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"]
Jul 28 19:43:42.773256 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:42.773234    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-7a6162-openshift-default-67b77b774d-jrhw2"]
Jul 28 19:43:43.143326 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:43.143295    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" path="/var/lib/kubelet/pods/d920e673-0200-4073-aaf0-026b1bf1f585/volumes"
Jul 28 19:43:46.242016 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:46.241980    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:43:54.728511 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:54.728473    2563 patch_prober.go:28] interesting pod/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 28 19:43:54.729040 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:54.728539    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 28 19:43:54.794460 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:54.794429    2563 generic.go:356] "Generic (PLEG): container finished" podID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerID="25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25" exitCode=0
Jul 28 19:43:54.794626 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:54.794503    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" event={"ID":"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d","Type":"ContainerDied","Data":"25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25"}
Jul 28 19:43:54.794933 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:54.794900    2563 scope.go:117] "RemoveContainer" containerID="25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25"
Jul 28 19:43:54.795876 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:54.795846    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:43:54.795984 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:54.795936    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:43:54.796071 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:54.795984    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:43:55.803070 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:55.803024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" event={"ID":"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d","Type":"ContainerStarted","Data":"08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c"}
Jul 28 19:43:56.255834 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:56.255803    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:43:56.256165 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:56.256145    2563 patch_prober.go:28] interesting pod/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 28 19:43:56.256216 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:56.256193    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 28 19:43:57.256032 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:57.255993    2563 patch_prober.go:28] interesting pod/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 28 19:43:57.256413 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:57.256059    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 28 19:43:58.256303 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:58.256266    2563 patch_prober.go:28] interesting pod/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 28 19:43:58.256685 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:58.256324    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 28 19:43:59.256167 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:59.256128    2563 patch_prober.go:28] interesting pod/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 28 19:43:59.256345 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:43:59.256194    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 28 19:44:00.259541 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:00.259509    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:44:00.259935 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:00.259852    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:44:00.260695 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:00.260676    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:44:05.573062 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:05.573026    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:05.907215 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:05.907128    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:06.172675 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.172593    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"]
Jul 28 19:44:06.173087 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.173067    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy"
Jul 28 19:44:06.173177 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.173088    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy"
Jul 28 19:44:06.173177 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.173108    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy"
Jul 28 19:44:06.173177 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.173117    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy"
Jul 28 19:44:06.173285 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.173189    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f7928d2-eb01-4a7c-bd22-1aad11a029b6" containerName="istio-proxy"
Jul 28 19:44:06.173285 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.173202    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy"
Jul 28 19:44:06.173285 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.173215    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d920e673-0200-4073-aaf0-026b1bf1f585" containerName="istio-proxy"
Jul 28 19:44:06.176453 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.176437    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.179121 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.179098    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-d66111-openshift-default-dockercfg-9np8c\""
Jul 28 19:44:06.186719 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.186697    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"]
Jul 28 19:44:06.199121 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.199093    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:06.290560 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.290515    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-credential-socket\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.290560 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.290551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-certs\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.290813 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.290601    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rtmx\" (UniqueName: \"kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-kube-api-access-8rtmx\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.290813 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.290658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-podinfo\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.290813 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.290693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-data\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.290813 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.290722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istiod-ca-cert\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.290813 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.290751    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-socket\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.290813 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.290780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-envoy\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.290813 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.290807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-token\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392148 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-podinfo\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392310 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-data\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392310 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istiod-ca-cert\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-socket\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392387 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-envoy\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392486 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-token\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392486 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-credential-socket\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392591 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-data\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392591 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-certs\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392591 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rtmx\" (UniqueName: \"kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-kube-api-access-8rtmx\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392716 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-socket\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.392990 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.392964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-credential-socket\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.393093 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.393055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-certs\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.393131 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.393092    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istiod-ca-cert\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.394853 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.394832    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-podinfo\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.394967 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.394887    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-envoy\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.400279 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.400254    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-token\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.400360 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.400324    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rtmx\" (UniqueName: \"kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-kube-api-access-8rtmx\") pod \"e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") " pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.470503 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.470406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:06.487284 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.487261    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:06.635015 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.634992    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"]
Jul 28 19:44:06.637789 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:44:06.637759    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd93cc4c7_a6c2_4583_b871_1686c457ec2f.slice/crio-cab4a9f32ecced2e9b2d681c363924a4b679d9d56b2ab4e5f69b7ff53aa4757a WatchSource:0}: Error finding container cab4a9f32ecced2e9b2d681c363924a4b679d9d56b2ab4e5f69b7ff53aa4757a: Status 404 returned error can't find the container with id cab4a9f32ecced2e9b2d681c363924a4b679d9d56b2ab4e5f69b7ff53aa4757a
Jul 28 19:44:06.640101 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.640066    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:44:06.640181 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.640145    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:44:06.640229 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.640184    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:44:06.856400 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.856362    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" event={"ID":"d93cc4c7-a6c2-4583-b871-1686c457ec2f","Type":"ContainerStarted","Data":"94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1"}
Jul 28 19:44:06.856578 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.856426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" event={"ID":"d93cc4c7-a6c2-4583-b871-1686c457ec2f","Type":"ContainerStarted","Data":"cab4a9f32ecced2e9b2d681c363924a4b679d9d56b2ab4e5f69b7ff53aa4757a"}
Jul 28 19:44:06.876742 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:06.876678    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" podStartSLOduration=0.876655647 podStartE2EDuration="876.655647ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:06.875450687 +0000 UTC m=+2488.337058403" watchObservedRunningTime="2026-07-28 19:44:06.876655647 +0000 UTC m=+2488.338263354"
Jul 28 19:44:07.487549 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:07.487517    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:07.492011 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:07.491978    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:07.860428 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:07.860400    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:07.861406 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:07.861383    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:08.791198 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:08.791161    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:08.891814 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:08.891780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:09.009879 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:09.009843    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:09.518547 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:09.518507    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:09.615180 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:09.615144    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:16.441278 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:16.441239    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:44:23.903157 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:23.903114    2563 patch_prober.go:28] interesting pod/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 19:44:23.903553 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:23.903196    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 19:44:24.944890 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:24.944857    2563 generic.go:356] "Generic (PLEG): container finished" podID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerID="94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1" exitCode=0
Jul 28 19:44:24.945320 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:24.944946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" event={"ID":"d93cc4c7-a6c2-4583-b871-1686c457ec2f","Type":"ContainerDied","Data":"94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1"}
Jul 28 19:44:24.945392 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:24.945378    2563 scope.go:117] "RemoveContainer" containerID="94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1"
Jul 28 19:44:24.946380 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:24.946357    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:44:24.946444 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:24.946426    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:44:24.946486 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:24.946463    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:44:25.950532 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:25.950496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" event={"ID":"d93cc4c7-a6c2-4583-b871-1686c457ec2f","Type":"ContainerStarted","Data":"747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3"}
Jul 28 19:44:26.488016 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:26.487983    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:26.488226 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:26.488208    2563 patch_prober.go:28] interesting pod/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 28 19:44:26.488278 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:26.488250    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 28 19:44:27.488411 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:27.488379    2563 patch_prober.go:28] interesting pod/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 28 19:44:27.488798 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:27.488431    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 28 19:44:28.487913 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:28.487872    2563 patch_prober.go:28] interesting pod/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 28 19:44:28.488115 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:28.487951    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 28 19:44:29.488273 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:29.488177    2563 patch_prober.go:28] interesting pod/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 28 19:44:29.488273 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:29.488229    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 28 19:44:30.491499 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:30.491469    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:30.491913 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:30.491726    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:30.492460 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:30.492443    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:44:33.693934 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:33.693893    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:48.700037 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:48.699998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:52.056633 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:44:52.056602    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:45:00.132982 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.132946    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j"]
Jul 28 19:45:00.136220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.136196    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr"]
Jul 28 19:45:00.136344 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.136307    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j"
Jul 28 19:45:00.139231 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.139208    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr"
Jul 28 19:45:00.139482 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.139465    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:45:00.146496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.146476    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j"]
Jul 28 19:45:00.147168 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.147128    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j"
Jul 28 19:45:00.149891 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.149872    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr"]
Jul 28 19:45:00.157121 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.157103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr"
Jul 28 19:45:00.303011 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.302838    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j"]
Jul 28 19:45:00.305450 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:45:00.305419    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8050dd95_8c4c_4ef6_9ca7_098e9450714d.slice/crio-6d8f2044f208dd327b79a67cbaa773dff7ea8b76932fb3de7f4a4853005380c5 WatchSource:0}: Error finding container 6d8f2044f208dd327b79a67cbaa773dff7ea8b76932fb3de7f4a4853005380c5: Status 404 returned error can't find the container with id 6d8f2044f208dd327b79a67cbaa773dff7ea8b76932fb3de7f4a4853005380c5
Jul 28 19:45:00.321149 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:00.321060    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr"]
Jul 28 19:45:00.323316 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:45:00.323290    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2036a29_7037_45dd_9069_1dfff2f7a542.slice/crio-b6fbe25a016ec5298d88d074d0313ab5d5520e01552a47228ebdb4375cf2910d WatchSource:0}: Error finding container b6fbe25a016ec5298d88d074d0313ab5d5520e01552a47228ebdb4375cf2910d: Status 404 returned error can't find the container with id b6fbe25a016ec5298d88d074d0313ab5d5520e01552a47228ebdb4375cf2910d
Jul 28 19:45:01.103239 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:01.103203    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr" event={"ID":"e2036a29-7037-45dd-9069-1dfff2f7a542","Type":"ContainerStarted","Data":"697399872f522b27688e7e5f658630da9c52f519165bc67cdec97970f78ffa4a"}
Jul 28 19:45:01.103437 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:01.103259    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr" event={"ID":"e2036a29-7037-45dd-9069-1dfff2f7a542","Type":"ContainerStarted","Data":"b6fbe25a016ec5298d88d074d0313ab5d5520e01552a47228ebdb4375cf2910d"}
Jul 28 19:45:01.104660 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:01.104630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j" event={"ID":"8050dd95-8c4c-4ef6-9ca7-098e9450714d","Type":"ContainerStarted","Data":"e57ab27a2994ab8d66038e4afe401fa707e8f34a2606d9a00a9b3fe7ee9d00ed"}
Jul 28 19:45:01.104781 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:01.104666    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j" event={"ID":"8050dd95-8c4c-4ef6-9ca7-098e9450714d","Type":"ContainerStarted","Data":"6d8f2044f208dd327b79a67cbaa773dff7ea8b76932fb3de7f4a4853005380c5"}
Jul 28 19:45:01.119515 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:01.119470    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr" podStartSLOduration=1.119453841 podStartE2EDuration="1.119453841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:45:01.118747943 +0000 UTC m=+2542.580355646" watchObservedRunningTime="2026-07-28 19:45:01.119453841 +0000 UTC m=+2542.581061545"
Jul 28 19:45:01.136146 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:01.136109    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j" podStartSLOduration=1.13609582 podStartE2EDuration="1.13609582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:45:01.1347581 +0000 UTC m=+2542.596365913" watchObservedRunningTime="2026-07-28 19:45:01.13609582 +0000 UTC m=+2542.597703542"
Jul 28 19:45:02.109846 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:02.109806    2563 generic.go:356] "Generic (PLEG): container finished" podID="8050dd95-8c4c-4ef6-9ca7-098e9450714d" containerID="e57ab27a2994ab8d66038e4afe401fa707e8f34a2606d9a00a9b3fe7ee9d00ed" exitCode=0
Jul 28 19:45:02.110025 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:02.109934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j" event={"ID":"8050dd95-8c4c-4ef6-9ca7-098e9450714d","Type":"ContainerDied","Data":"e57ab27a2994ab8d66038e4afe401fa707e8f34a2606d9a00a9b3fe7ee9d00ed"}
Jul 28 19:45:02.111419 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:02.111391    2563 generic.go:356] "Generic (PLEG): container finished" podID="e2036a29-7037-45dd-9069-1dfff2f7a542" containerID="697399872f522b27688e7e5f658630da9c52f519165bc67cdec97970f78ffa4a" exitCode=0
Jul 28 19:45:02.111548 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:02.111451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr" event={"ID":"e2036a29-7037-45dd-9069-1dfff2f7a542","Type":"ContainerDied","Data":"697399872f522b27688e7e5f658630da9c52f519165bc67cdec97970f78ffa4a"}
Jul 28 19:45:03.262759 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:03.262738    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr"
Jul 28 19:45:03.265557 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:03.265538    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j"
Jul 28 19:45:04.120983 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:04.120949    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr"
Jul 28 19:45:04.121187 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:04.120979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr" event={"ID":"e2036a29-7037-45dd-9069-1dfff2f7a542","Type":"ContainerDied","Data":"b6fbe25a016ec5298d88d074d0313ab5d5520e01552a47228ebdb4375cf2910d"}
Jul 28 19:45:04.121187 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:04.121036    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6fbe25a016ec5298d88d074d0313ab5d5520e01552a47228ebdb4375cf2910d"
Jul 28 19:45:04.122276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:04.122256    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j"
Jul 28 19:45:04.122416 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:04.122256    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j" event={"ID":"8050dd95-8c4c-4ef6-9ca7-098e9450714d","Type":"ContainerDied","Data":"6d8f2044f208dd327b79a67cbaa773dff7ea8b76932fb3de7f4a4853005380c5"}
Jul 28 19:45:04.122416 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:04.122384    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d8f2044f208dd327b79a67cbaa773dff7ea8b76932fb3de7f4a4853005380c5"
Jul 28 19:45:18.096797 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:18.096744    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:23.655979 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.655940    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"]
Jul 28 19:45:23.656381 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.656238    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2036a29-7037-45dd-9069-1dfff2f7a542" containerName="cleanup"
Jul 28 19:45:23.656381 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.656248    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2036a29-7037-45dd-9069-1dfff2f7a542" containerName="cleanup"
Jul 28 19:45:23.656381 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.656258    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8050dd95-8c4c-4ef6-9ca7-098e9450714d" containerName="cleanup"
Jul 28 19:45:23.656381 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.656263    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8050dd95-8c4c-4ef6-9ca7-098e9450714d" containerName="cleanup"
Jul 28 19:45:23.656381 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.656313    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8050dd95-8c4c-4ef6-9ca7-098e9450714d" containerName="cleanup"
Jul 28 19:45:23.656381 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.656323    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2036a29-7037-45dd-9069-1dfff2f7a542" containerName="cleanup"
Jul 28 19:45:23.659079 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.659064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.663116 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.663091    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"openshift-service-ca.crt\""
Jul 28 19:45:23.663262 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.663089    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"kube-root-ca.crt\""
Jul 28 19:45:23.663262 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.663092    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"default-dockercfg-4fkhj\""
Jul 28 19:45:23.663262 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.663097    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"auth-test-model-d66111-kserve-self-signed-certs\""
Jul 28 19:45:23.667519 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.667498    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"]
Jul 28 19:45:23.750124 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.750087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:23.779145 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.779115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tls-certs\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.779266 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.779149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tmp-dir\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.779266 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.779179    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-home\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.779338 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.779275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-model-cache\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.779338 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.779305    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-dshm\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.779338 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.779322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2vcx\" (UniqueName: \"kubernetes.io/projected/b7e7233a-a04a-41d1-be69-8e23ad68ba29-kube-api-access-b2vcx\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.880528 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.880493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-model-cache\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.880528 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.880533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-dshm\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.880766 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.880549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2vcx\" (UniqueName: \"kubernetes.io/projected/b7e7233a-a04a-41d1-be69-8e23ad68ba29-kube-api-access-b2vcx\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.880766 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.880590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tls-certs\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.880766 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.880722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tmp-dir\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.880953 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.880786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-home\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.880953 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.880858    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-model-cache\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.881077 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.881045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tmp-dir\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.881166 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.881146    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-home\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.883043 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.883016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-dshm\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.883298 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.883281    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tls-certs\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.890720 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.890695    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2vcx\" (UniqueName: \"kubernetes.io/projected/b7e7233a-a04a-41d1-be69-8e23ad68ba29-kube-api-access-b2vcx\") pod \"auth-test-model-d66111-kserve-d5cb4fbc-h8qz2\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") " pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:23.968981 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:23.968863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:24.108418 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:24.108392    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"]
Jul 28 19:45:24.110832 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:45:24.110809    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7e7233a_a04a_41d1_be69_8e23ad68ba29.slice/crio-db79e83b0cc56a4cad214a962638214a5fa825c0f6dc79646567d567f5fcea35 WatchSource:0}: Error finding container db79e83b0cc56a4cad214a962638214a5fa825c0f6dc79646567d567f5fcea35: Status 404 returned error can't find the container with id db79e83b0cc56a4cad214a962638214a5fa825c0f6dc79646567d567f5fcea35
Jul 28 19:45:24.205513 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:24.205476    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2" event={"ID":"b7e7233a-a04a-41d1-be69-8e23ad68ba29","Type":"ContainerStarted","Data":"7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52"}
Jul 28 19:45:24.205513 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:24.205521    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2" event={"ID":"b7e7233a-a04a-41d1-be69-8e23ad68ba29","Type":"ContainerStarted","Data":"db79e83b0cc56a4cad214a962638214a5fa825c0f6dc79646567d567f5fcea35"}
Jul 28 19:45:24.223692 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:24.223586    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2" podStartSLOduration=1.223569366 podStartE2EDuration="1.223569366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:45:24.222255946 +0000 UTC m=+2565.683863649" watchObservedRunningTime="2026-07-28 19:45:24.223569366 +0000 UTC m=+2565.685177069"
Jul 28 19:45:27.638981 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:27.638941    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:45:33.969105 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:33.969062    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:33.969514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:33.969122    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:33.981721 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:33.981690    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:34.260347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:34.260265    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:45:38.670122 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:38.670091    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:52.533732 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:45:52.533689    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:46:18.537694 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:46:18.537602    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:46:36.545608 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:46:36.545572    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:46:48.962596 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:46:48.962541    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:46:58.148691 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:46:58.148653    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:47:15.738518 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:47:15.738485    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:47:35.938663 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:47:35.938583    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:47:39.138771 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:47:39.138742    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:47:39.143687 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:47:39.143667    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:47:54.036611 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:47:54.036569    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:48:16.335017 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:48:16.334980    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:48:18.533514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:48:18.533475    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:34.435866 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:48:34.435828    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:48:57.043081 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:48:57.043048    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:49:15.243505 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:15.243420    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:49:37.947951 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:37.946051    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:49:41.721191 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:41.721141    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:42.632651 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:42.632616    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"]
Jul 28 19:49:42.710481 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:42.710449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:51.681348 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:51.681309    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:54.143136 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:54.143099    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:49:57.651567 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.651528    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2" podUID="b7e7233a-a04a-41d1-be69-8e23ad68ba29" containerName="main" containerID="cri-o://7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52" gracePeriod=45
Jul 28 19:49:57.901354 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.901329    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:49:57.949946 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.949842    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-home\") pod \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") "
Jul 28 19:49:57.949946 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.949893    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tmp-dir\") pod \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") "
Jul 28 19:49:57.949946 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.949914    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-model-cache\") pod \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") "
Jul 28 19:49:57.950220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.949981    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tls-certs\") pod \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") "
Jul 28 19:49:57.950220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.950007    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2vcx\" (UniqueName: \"kubernetes.io/projected/b7e7233a-a04a-41d1-be69-8e23ad68ba29-kube-api-access-b2vcx\") pod \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") "
Jul 28 19:49:57.950220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.950027    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-dshm\") pod \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\" (UID: \"b7e7233a-a04a-41d1-be69-8e23ad68ba29\") "
Jul 28 19:49:57.950220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.950201    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b7e7233a-a04a-41d1-be69-8e23ad68ba29" (UID: "b7e7233a-a04a-41d1-be69-8e23ad68ba29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:49:57.950220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.950211    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-home" (OuterVolumeSpecName: "home") pod "b7e7233a-a04a-41d1-be69-8e23ad68ba29" (UID: "b7e7233a-a04a-41d1-be69-8e23ad68ba29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:49:57.950402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.950233    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-model-cache" (OuterVolumeSpecName: "model-cache") pod "b7e7233a-a04a-41d1-be69-8e23ad68ba29" (UID: "b7e7233a-a04a-41d1-be69-8e23ad68ba29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:49:57.950402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.950278    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-home\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:49:57.950402 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.950295    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tmp-dir\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:49:57.952408 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.952381    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b7e7233a-a04a-41d1-be69-8e23ad68ba29" (UID: "b7e7233a-a04a-41d1-be69-8e23ad68ba29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:49:57.952506 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.952441    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7e7233a-a04a-41d1-be69-8e23ad68ba29-kube-api-access-b2vcx" (OuterVolumeSpecName: "kube-api-access-b2vcx") pod "b7e7233a-a04a-41d1-be69-8e23ad68ba29" (UID: "b7e7233a-a04a-41d1-be69-8e23ad68ba29"). InnerVolumeSpecName "kube-api-access-b2vcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:49:57.952506 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:57.952456    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-dshm" (OuterVolumeSpecName: "dshm") pod "b7e7233a-a04a-41d1-be69-8e23ad68ba29" (UID: "b7e7233a-a04a-41d1-be69-8e23ad68ba29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:49:58.051647 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.051597    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7e7233a-a04a-41d1-be69-8e23ad68ba29-tls-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:49:58.051647 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.051641    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2vcx\" (UniqueName: \"kubernetes.io/projected/b7e7233a-a04a-41d1-be69-8e23ad68ba29-kube-api-access-b2vcx\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:49:58.051647 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.051650    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-dshm\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:49:58.051647 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.051659    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7e7233a-a04a-41d1-be69-8e23ad68ba29-model-cache\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:49:58.407306 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.407272    2563 generic.go:356] "Generic (PLEG): container finished" podID="b7e7233a-a04a-41d1-be69-8e23ad68ba29" containerID="7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52" exitCode=0
Jul 28 19:49:58.407502 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.407313    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"
Jul 28 19:49:58.407502 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.407326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2" event={"ID":"b7e7233a-a04a-41d1-be69-8e23ad68ba29","Type":"ContainerDied","Data":"7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52"}
Jul 28 19:49:58.407502 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.407366    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2" event={"ID":"b7e7233a-a04a-41d1-be69-8e23ad68ba29","Type":"ContainerDied","Data":"db79e83b0cc56a4cad214a962638214a5fa825c0f6dc79646567d567f5fcea35"}
Jul 28 19:49:58.407502 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.407381    2563 scope.go:117] "RemoveContainer" containerID="7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52"
Jul 28 19:49:58.416855 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.416825    2563 scope.go:117] "RemoveContainer" containerID="7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52"
Jul 28 19:49:58.417228 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:49:58.417208    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52\": container with ID starting with 7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52 not found: ID does not exist" containerID="7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52"
Jul 28 19:49:58.417317 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.417240    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52"} err="failed to get container status \"7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52\": rpc error: code = NotFound desc = could not find container \"7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52\": container with ID starting with 7f1b2c9e1b93632515483efc6ecfc668ea6bd8fa95c803a73ec5522962863f52 not found: ID does not exist"
Jul 28 19:49:58.428192 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.428162    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"]
Jul 28 19:49:58.433169 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:58.433145    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/auth-test-model-d66111-kserve-d5cb4fbc-h8qz2"]
Jul 28 19:49:59.143669 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:49:59.143635    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7e7233a-a04a-41d1-be69-8e23ad68ba29" path="/var/lib/kubelet/pods/b7e7233a-a04a-41d1-be69-8e23ad68ba29/volumes"
Jul 28 19:50:21.582569 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:50:21.582527    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:24.950880 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:50:24.950835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:51:00.621207 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:00.621169    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:00.950450 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:00.950363    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:51:12.732355 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:12.732291    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:12.833713 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:12.833681    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:21.965410 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:21.965378    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:51:27.379633 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:27.379598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:28.321775 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:28.321739    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:32.058025 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.057987    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"]
Jul 28 19:51:32.058454 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.058307    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e7233a-a04a-41d1-be69-8e23ad68ba29" containerName="main"
Jul 28 19:51:32.058454 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.058320    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e7233a-a04a-41d1-be69-8e23ad68ba29" containerName="main"
Jul 28 19:51:32.058454 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.058396    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e7233a-a04a-41d1-be69-8e23ad68ba29" containerName="main"
Jul 28 19:51:32.061209 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.061194    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.064263 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.064241    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ec81e7\"/\"default-dockercfg-442dt\""
Jul 28 19:51:32.065443 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.065420    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ec81e7\"/\"kube-root-ca.crt\""
Jul 28 19:51:32.065548 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.065424    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ec81e7\"/\"rate-test-model-ec81e7-kserve-self-signed-certs\""
Jul 28 19:51:32.065548 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.065430    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ec81e7\"/\"openshift-service-ca.crt\""
Jul 28 19:51:32.072836 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.072812    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"]
Jul 28 19:51:32.162354 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.162319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-home\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.162548 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.162363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-tmp-dir\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.162548 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.162467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmw8f\" (UniqueName: \"kubernetes.io/projected/a51fac66-b78f-4600-8316-6ef214fa8824-kube-api-access-lmw8f\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.162548 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.162503    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:32.162548 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.162529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51fac66-b78f-4600-8316-6ef214fa8824-tls-certs\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.162778 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.162673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-model-cache\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.162778 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.162715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-dshm\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.264026 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.263989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-model-cache\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.264026 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.264023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-dshm\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.264254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.264051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-home\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.264254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.264073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-tmp-dir\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.264254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.264105    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmw8f\" (UniqueName: \"kubernetes.io/projected/a51fac66-b78f-4600-8316-6ef214fa8824-kube-api-access-lmw8f\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.264254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.264137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51fac66-b78f-4600-8316-6ef214fa8824-tls-certs\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.264516 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.264384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-home\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.264516 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.264442    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-model-cache\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.264622 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.264556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-tmp-dir\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.266675 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.266644    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-dshm\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.266896 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.266874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51fac66-b78f-4600-8316-6ef214fa8824-tls-certs\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.272242 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.272224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmw8f\" (UniqueName: \"kubernetes.io/projected/a51fac66-b78f-4600-8316-6ef214fa8824-kube-api-access-lmw8f\") pod \"rate-test-model-ec81e7-kserve-b5878b759-8srzj\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") " pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.370247 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.370199    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:32.509585 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.509555    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"]
Jul 28 19:51:32.512037 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:51:32.511998    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda51fac66_b78f_4600_8316_6ef214fa8824.slice/crio-665039a78a6b957cd0930184617b4917f2188a46053ed0354c81bbe1af0ae784 WatchSource:0}: Error finding container 665039a78a6b957cd0930184617b4917f2188a46053ed0354c81bbe1af0ae784: Status 404 returned error can't find the container with id 665039a78a6b957cd0930184617b4917f2188a46053ed0354c81bbe1af0ae784
Jul 28 19:51:32.823445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.819953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj" event={"ID":"a51fac66-b78f-4600-8316-6ef214fa8824","Type":"ContainerStarted","Data":"fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd"}
Jul 28 19:51:32.823445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.819999    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj" event={"ID":"a51fac66-b78f-4600-8316-6ef214fa8824","Type":"ContainerStarted","Data":"665039a78a6b957cd0930184617b4917f2188a46053ed0354c81bbe1af0ae784"}
Jul 28 19:51:32.840111 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:32.840044    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj" podStartSLOduration=0.840022704 podStartE2EDuration="840.022704ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:51:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:51:32.838127001 +0000 UTC m=+2934.299734704" watchObservedRunningTime="2026-07-28 19:51:32.840022704 +0000 UTC m=+2934.301630411"
Jul 28 19:51:40.247347 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:40.247309    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:51:42.371225 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:42.371185    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:42.371225 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:42.371237    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:42.383737 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:42.383711    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:42.875799 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:42.875773    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:51:51.369991 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:51.369959    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:55.256729 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:51:55.256695    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:52:07.255464 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:52:07.255378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:30.947688 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:52:30.947653    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:52:39.163909 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:52:39.163884    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:52:39.169556 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:52:39.169535    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:52:48.747140 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:52:48.747100    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:53:10.139624 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:10.139579    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:53:16.016540 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:16.016500    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"]
Jul 28 19:53:16.037135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:16.037105    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:16.979463 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:16.979428    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:24.436287 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:24.436248    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:53:31.037450 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.037345    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj" podUID="a51fac66-b78f-4600-8316-6ef214fa8824" containerName="main" containerID="cri-o://fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd" gracePeriod=45
Jul 28 19:53:31.289747 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.289684    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:53:31.341435 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.341397    2563 generic.go:356] "Generic (PLEG): container finished" podID="a51fac66-b78f-4600-8316-6ef214fa8824" containerID="fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd" exitCode=0
Jul 28 19:53:31.341626 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.341445    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"
Jul 28 19:53:31.341626 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.341481    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj" event={"ID":"a51fac66-b78f-4600-8316-6ef214fa8824","Type":"ContainerDied","Data":"fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd"}
Jul 28 19:53:31.341626 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.341522    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj" event={"ID":"a51fac66-b78f-4600-8316-6ef214fa8824","Type":"ContainerDied","Data":"665039a78a6b957cd0930184617b4917f2188a46053ed0354c81bbe1af0ae784"}
Jul 28 19:53:31.341626 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.341540    2563 scope.go:117] "RemoveContainer" containerID="fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd"
Jul 28 19:53:31.349695 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.349678    2563 scope.go:117] "RemoveContainer" containerID="fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd"
Jul 28 19:53:31.350017 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:53:31.349997    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd\": container with ID starting with fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd not found: ID does not exist" containerID="fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd"
Jul 28 19:53:31.350093 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.350026    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd"} err="failed to get container status \"fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd\": rpc error: code = NotFound desc = could not find container \"fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd\": container with ID starting with fc01044dfa72755a3445739679214485533507b66dd981461892855a73dfb8dd not found: ID does not exist"
Jul 28 19:53:31.472439 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.472405    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-tmp-dir\") pod \"a51fac66-b78f-4600-8316-6ef214fa8824\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") "
Jul 28 19:53:31.472615 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.472474    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-dshm\") pod \"a51fac66-b78f-4600-8316-6ef214fa8824\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") "
Jul 28 19:53:31.472615 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.472499    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51fac66-b78f-4600-8316-6ef214fa8824-tls-certs\") pod \"a51fac66-b78f-4600-8316-6ef214fa8824\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") "
Jul 28 19:53:31.472615 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.472539    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-home\") pod \"a51fac66-b78f-4600-8316-6ef214fa8824\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") "
Jul 28 19:53:31.472615 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.472566    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmw8f\" (UniqueName: \"kubernetes.io/projected/a51fac66-b78f-4600-8316-6ef214fa8824-kube-api-access-lmw8f\") pod \"a51fac66-b78f-4600-8316-6ef214fa8824\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") "
Jul 28 19:53:31.472615 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.472591    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-model-cache\") pod \"a51fac66-b78f-4600-8316-6ef214fa8824\" (UID: \"a51fac66-b78f-4600-8316-6ef214fa8824\") "
Jul 28 19:53:31.472878 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.472732    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a51fac66-b78f-4600-8316-6ef214fa8824" (UID: "a51fac66-b78f-4600-8316-6ef214fa8824"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:53:31.472878 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.472821    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-home" (OuterVolumeSpecName: "home") pod "a51fac66-b78f-4600-8316-6ef214fa8824" (UID: "a51fac66-b78f-4600-8316-6ef214fa8824"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:53:31.473526 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.473440    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-model-cache" (OuterVolumeSpecName: "model-cache") pod "a51fac66-b78f-4600-8316-6ef214fa8824" (UID: "a51fac66-b78f-4600-8316-6ef214fa8824"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:53:31.480511 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.480475    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-dshm" (OuterVolumeSpecName: "dshm") pod "a51fac66-b78f-4600-8316-6ef214fa8824" (UID: "a51fac66-b78f-4600-8316-6ef214fa8824"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:53:31.480690 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.480669    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a51fac66-b78f-4600-8316-6ef214fa8824-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a51fac66-b78f-4600-8316-6ef214fa8824" (UID: "a51fac66-b78f-4600-8316-6ef214fa8824"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:53:31.480690 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.480672    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a51fac66-b78f-4600-8316-6ef214fa8824-kube-api-access-lmw8f" (OuterVolumeSpecName: "kube-api-access-lmw8f") pod "a51fac66-b78f-4600-8316-6ef214fa8824" (UID: "a51fac66-b78f-4600-8316-6ef214fa8824"). InnerVolumeSpecName "kube-api-access-lmw8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:53:31.573497 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.573461    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-home\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:53:31.573669 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.573520    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmw8f\" (UniqueName: \"kubernetes.io/projected/a51fac66-b78f-4600-8316-6ef214fa8824-kube-api-access-lmw8f\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:53:31.573669 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.573536    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-model-cache\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:53:31.573669 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.573548    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-tmp-dir\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:53:31.573669 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.573558    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51fac66-b78f-4600-8316-6ef214fa8824-dshm\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:53:31.573669 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.573568    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51fac66-b78f-4600-8316-6ef214fa8824-tls-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:53:31.664549 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.664513    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"]
Jul 28 19:53:31.667272 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:31.667249    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/rate-test-model-ec81e7-kserve-b5878b759-8srzj"]
Jul 28 19:53:33.143256 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:33.143223    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a51fac66-b78f-4600-8316-6ef214fa8824" path="/var/lib/kubelet/pods/a51fac66-b78f-4600-8316-6ef214fa8824/volumes"
Jul 28 19:53:36.883605 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:36.883568    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:37.171746 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.171666    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"]
Jul 28 19:53:37.172014 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.172000    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a51fac66-b78f-4600-8316-6ef214fa8824" containerName="main"
Jul 28 19:53:37.172068 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.172019    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a51fac66-b78f-4600-8316-6ef214fa8824" containerName="main"
Jul 28 19:53:37.172104 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.172094    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a51fac66-b78f-4600-8316-6ef214fa8824" containerName="main"
Jul 28 19:53:37.176508 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.176489    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.179295 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.179270    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-cf1393-openshift-default-dockercfg-xmjz8\""
Jul 28 19:53:37.185747 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.185721    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"]
Jul 28 19:53:37.195135 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.195107    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:37.324083 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.324047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-certs\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.324236 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.324098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-credential-socket\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.324236 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.324144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-envoy\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.324236 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.324162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51e47b57-89ed-4b88-8af4-836785e20af8-istio-podinfo\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.324236 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.324208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-socket\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.324395 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.324237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-istio-token\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.324395 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.324256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51e47b57-89ed-4b88-8af4-836785e20af8-istiod-ca-cert\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.324395 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.324274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd4zh\" (UniqueName: \"kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-kube-api-access-bd4zh\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.324395 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.324340    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-data\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.424954 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.424849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-socket\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.424954 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.424888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-istio-token\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.424954 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.424907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51e47b57-89ed-4b88-8af4-836785e20af8-istiod-ca-cert\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425208 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.424967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd4zh\" (UniqueName: \"kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-kube-api-access-bd4zh\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425208 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.425001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-data\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425208 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.425166    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-certs\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425364 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.425240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-credential-socket\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425364 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.425271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-envoy\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425364 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.425294    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-socket\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425364 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.425295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51e47b57-89ed-4b88-8af4-836785e20af8-istio-podinfo\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425576 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.425384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-data\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425639 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.425582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-credential-socket\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425639 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.425617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-certs\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.425728 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.425688    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51e47b57-89ed-4b88-8af4-836785e20af8-istiod-ca-cert\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.427433 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.427415    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-envoy\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.427742 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.427726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51e47b57-89ed-4b88-8af4-836785e20af8-istio-podinfo\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.433015 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.432996    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-istio-token\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.433384 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.433363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd4zh\" (UniqueName: \"kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-kube-api-access-bd4zh\") pod \"e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") " pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.487893 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.487847    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:37.630298 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.630271    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"]
Jul 28 19:53:37.632597 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:53:37.632565    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e47b57_89ed_4b88_8af4_836785e20af8.slice/crio-4882af6f4b9ad2024539d0e0480b71175cc89b0b084b9fe6d1033be60e023e3a WatchSource:0}: Error finding container 4882af6f4b9ad2024539d0e0480b71175cc89b0b084b9fe6d1033be60e023e3a: Status 404 returned error can't find the container with id 4882af6f4b9ad2024539d0e0480b71175cc89b0b084b9fe6d1033be60e023e3a
Jul 28 19:53:37.634705 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.634672    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:53:37.634802 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.634735    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:53:37.634802 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.634763    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:53:37.747488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:37.747450    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:38.375006 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:38.374970    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" event={"ID":"51e47b57-89ed-4b88-8af4-836785e20af8","Type":"ContainerStarted","Data":"cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83"}
Jul 28 19:53:38.375382 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:38.375024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" event={"ID":"51e47b57-89ed-4b88-8af4-836785e20af8","Type":"ContainerStarted","Data":"4882af6f4b9ad2024539d0e0480b71175cc89b0b084b9fe6d1033be60e023e3a"}
Jul 28 19:53:38.394571 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:38.394520    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" podStartSLOduration=1.394507817 podStartE2EDuration="1.394507817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:53:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:53:38.393898369 +0000 UTC m=+3059.855506073" watchObservedRunningTime="2026-07-28 19:53:38.394507817 +0000 UTC m=+3059.856115520"
Jul 28 19:53:38.488523 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:38.488485    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:38.493182 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:38.493158    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:39.379679 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:39.379645    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:39.380683 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:39.380660    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:40.366788 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:40.366756    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:40.450438 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:40.450402    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:40.534846 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:40.534802    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:40.870879 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:40.870847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:41.160240 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:41.160162    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:41.231473 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:41.231446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:45.166039 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:45.165999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:49.041166 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:49.041133    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:53:57.479091 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:57.479056    2563 generic.go:356] "Generic (PLEG): container finished" podID="51e47b57-89ed-4b88-8af4-836785e20af8" containerID="cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83" exitCode=0
Jul 28 19:53:57.479514 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:57.479134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" event={"ID":"51e47b57-89ed-4b88-8af4-836785e20af8","Type":"ContainerDied","Data":"cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83"}
Jul 28 19:53:57.479563 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:57.479525    2563 scope.go:117] "RemoveContainer" containerID="cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83"
Jul 28 19:53:57.480542 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:57.480516    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:53:57.480606 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:57.480580    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:53:57.480646 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:57.480619    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:53:58.485482 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:58.485450    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" event={"ID":"51e47b57-89ed-4b88-8af4-836785e20af8","Type":"ContainerStarted","Data":"bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84"}
Jul 28 19:53:58.489036 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:58.489016    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:53:58.489205 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:58.489183    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 28 19:53:58.489276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:58.489236    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 28 19:53:59.488558 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:59.488527    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 28 19:53:59.489007 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:53:59.488580    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 28 19:54:00.488981 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:00.488945    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 28 19:54:00.489383 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:00.489000    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 28 19:54:01.488203 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:01.488170    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 28 19:54:01.488386 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:01.488225    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 28 19:54:02.488520 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:02.488478    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 28 19:54:02.488956 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:02.488537    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 28 19:54:03.492253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:03.492218    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:54:03.492636 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:03.492500    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:54:03.493189 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:03.493170    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:54:05.357686 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:05.357644    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:06.118132 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:06.118097    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:06.320451 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:06.320414    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:06.697765 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:06.697731    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:09.586906 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:09.586875    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:09.687317 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:09.687286    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:10.953220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:10.953178    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:11.068379 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:11.068343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:11.410661 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:11.410620    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:11.508428 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:11.508390    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:19.044703 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:19.044667    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:54:24.228408 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:24.228374    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:39.239474 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.239434    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"]
Jul 28 19:54:39.242823 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.242805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.245580 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.245555    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-cf1393\"/\"openshift-service-ca.crt\""
Jul 28 19:54:39.245691 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.245578    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-cf1393\"/\"kube-root-ca.crt\""
Jul 28 19:54:39.246712 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.246688    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-cf1393\"/\"default-dockercfg-vzwhz\""
Jul 28 19:54:39.246849 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.246831    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-cf1393\"/\"ipp-route-cf1393-kserve-self-signed-certs\""
Jul 28 19:54:39.251395 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.251375    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"]
Jul 28 19:54:39.261280 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.261251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tls-certs\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.261404 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.261296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-dshm\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.261404 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.261336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-home\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.261404 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.261397    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tmp-dir\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.261536 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.261428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-model-cache\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.261536 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.261472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4s6d\" (UniqueName: \"kubernetes.io/projected/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-kube-api-access-p4s6d\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.340617 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.340584    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:39.362595 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.362566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-model-cache\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.362707 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.362606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4s6d\" (UniqueName: \"kubernetes.io/projected/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-kube-api-access-p4s6d\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.362707 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.362634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tls-certs\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.362785 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.362755    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-dshm\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.362829 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.362797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-home\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.362888 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.362872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tmp-dir\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.363057 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.363040    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-model-cache\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.363261 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.363240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tmp-dir\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.363319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.363256    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-home\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.365251 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.365229    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-dshm\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.365431 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.365417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tls-certs\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.372981 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.372960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4s6d\" (UniqueName: \"kubernetes.io/projected/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-kube-api-access-p4s6d\") pod \"ipp-route-cf1393-kserve-6b7495c68c-d26dh\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") " pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.552778 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.552695    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:39.685350 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:39.685313    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"]
Jul 28 19:54:39.688456 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:54:39.688431    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3844be21_4eb4_4ac7_aa2e_cbbed11c9790.slice/crio-b49973e9c56537faf2812f225b26a576a00056e8130eca749620e56ff80442ff WatchSource:0}: Error finding container b49973e9c56537faf2812f225b26a576a00056e8130eca749620e56ff80442ff: Status 404 returned error can't find the container with id b49973e9c56537faf2812f225b26a576a00056e8130eca749620e56ff80442ff
Jul 28 19:54:40.676620 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:40.676583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh" event={"ID":"3844be21-4eb4-4ac7-aa2e-cbbed11c9790","Type":"ContainerStarted","Data":"65ab89a10f3ed63fd9207adfbadb3638267c5859ae0d5c14a2125b3836a9f982"}
Jul 28 19:54:40.677025 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:40.676628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh" event={"ID":"3844be21-4eb4-4ac7-aa2e-cbbed11c9790","Type":"ContainerStarted","Data":"b49973e9c56537faf2812f225b26a576a00056e8130eca749620e56ff80442ff"}
Jul 28 19:54:40.693493 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:40.693447    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh" podStartSLOduration=1.693431873 podStartE2EDuration="1.693431873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:54:40.691713403 +0000 UTC m=+3122.153321105" watchObservedRunningTime="2026-07-28 19:54:40.693431873 +0000 UTC m=+3122.155039575"
Jul 28 19:54:41.941329 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:41.941288    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:54:49.552930 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:49.552875    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:49.553381 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:49.552953    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:49.565411 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:49.565382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:54:49.728027 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:54:49.728000    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:55:17.146825 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:17.146734    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:55:29.127417 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.127372    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:29.414491 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.414415    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"]
Jul 28 19:55:29.417699 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.417682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.420638 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.420616    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-ec8cba-openshift-default-dockercfg-tqq2g\""
Jul 28 19:55:29.427862 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.427839    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"]
Jul 28 19:55:29.436971 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.436948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:29.495825 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.495780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-certs\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.496040 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.495945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-envoy\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.496040 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.495977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-token\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.496040 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.496031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-podinfo\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.496193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.496067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-credential-socket\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.496193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.496102    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb744\" (UniqueName: \"kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-kube-api-access-bb744\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.496193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.496132    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-data\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.496193 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.496166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-socket\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.496433 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.496200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.596734 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.596688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-envoy\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.596734 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.596732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-token\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.596763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-podinfo\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.596785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-credential-socket\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.596803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb744\" (UniqueName: \"kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-kube-api-access-bb744\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.596820    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-data\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.596840    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-socket\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597024 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.596859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597331 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.597050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-certs\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597331 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.597233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-credential-socket\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597331 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.597263    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-data\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597489 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.597328    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-socket\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597489 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.597403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-certs\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.597733 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.597717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.599396 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.599375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-podinfo\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.599496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.599445    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-envoy\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.604551 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.604532    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-token\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.604675 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.604653    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb744\" (UniqueName: \"kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-kube-api-access-bb744\") pod \"e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") " pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.728996 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.728888    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:29.748762 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.748734    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:29.875732 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.875706    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"]
Jul 28 19:55:29.877949 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:55:29.877906    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e84b594_b5df_4e1a_92d7_a8b1d7b30026.slice/crio-4b5779b3ccf29a3f9c1ffe8256a72cde78864c2001d333cff31ef44381112ff6 WatchSource:0}: Error finding container 4b5779b3ccf29a3f9c1ffe8256a72cde78864c2001d333cff31ef44381112ff6: Status 404 returned error can't find the container with id 4b5779b3ccf29a3f9c1ffe8256a72cde78864c2001d333cff31ef44381112ff6
Jul 28 19:55:29.880125 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.880095    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:55:29.880222 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.880161    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:55:29.880222 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.880188    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:55:29.892988 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:29.892967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" event={"ID":"5e84b594-b5df-4e1a-92d7-a8b1d7b30026","Type":"ContainerStarted","Data":"4b5779b3ccf29a3f9c1ffe8256a72cde78864c2001d333cff31ef44381112ff6"}
Jul 28 19:55:30.898819 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:30.898780    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" event={"ID":"5e84b594-b5df-4e1a-92d7-a8b1d7b30026","Type":"ContainerStarted","Data":"b233f688caadd64305e31b9f7b1c41af0c8e9945f4ffabca65365b543e5667f3"}
Jul 28 19:55:30.921122 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:30.921076    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" podStartSLOduration=1.921062072 podStartE2EDuration="1.921062072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:55:30.919974118 +0000 UTC m=+3172.381581822" watchObservedRunningTime="2026-07-28 19:55:30.921062072 +0000 UTC m=+3172.382669775"
Jul 28 19:55:31.729531 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:31.729495    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:31.734527 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:31.734505    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:31.903073 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:31.903043    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:31.904046 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:31.904023    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:32.896569 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:32.896538    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:32.993458 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:32.993421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:34.405468 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:34.405436    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:34.505897 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:34.505869    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:34.896487 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:34.896449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:34.988558 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:34.988526    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:43.441351 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:43.441310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:55:47.530846 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:47.530798    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:51.999883 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:51.999851    2563 generic.go:356] "Generic (PLEG): container finished" podID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerID="b233f688caadd64305e31b9f7b1c41af0c8e9945f4ffabca65365b543e5667f3" exitCode=0
Jul 28 19:55:52.000309 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:51.999946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" event={"ID":"5e84b594-b5df-4e1a-92d7-a8b1d7b30026","Type":"ContainerDied","Data":"b233f688caadd64305e31b9f7b1c41af0c8e9945f4ffabca65365b543e5667f3"}
Jul 28 19:55:52.000445 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:52.000428    2563 scope.go:117] "RemoveContainer" containerID="b233f688caadd64305e31b9f7b1c41af0c8e9945f4ffabca65365b543e5667f3"
Jul 28 19:55:52.001855 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:52.001824    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:55:52.001964 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:52.001914    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:55:52.002015 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:52.002000    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 19:55:53.010621 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:53.010580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" event={"ID":"5e84b594-b5df-4e1a-92d7-a8b1d7b30026","Type":"ContainerStarted","Data":"265b28e58567a1c6de4dd23f7b7d45f4cc657381cf98160982bd3902b03022f4"}
Jul 28 19:55:53.729568 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:53.729531    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:53.729796 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:53.729774    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 19:55:53.729850 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:53.729819    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 19:55:54.729875 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:54.729834    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 19:55:54.730292 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:54.729900    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 19:55:55.729719 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:55.729683    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 19:55:55.729904 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:55.729738    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 19:55:55.790429 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:55.790397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:55.856434 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:55.856404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:55.956809 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:55.956770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:56.057795 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:56.057700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:56.729235 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:56.729196    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 28 19:55:56.729427 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:56.729269    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 28 19:55:57.384119 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:57.384076    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-64.ec2.internal" podUID="20fec549702e81f5927fb23efca53227" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Jul 28 19:55:57.522269 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:57.522233    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:57.734016 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:57.733913    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:57.734318 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:57.734297    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:55:57.735009 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:55:57.734991    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:56:03.703627 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:03.703593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:03.820468 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:03.820432    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:03.937729 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:03.937695    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"]
Jul 28 19:56:03.937990 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:03.937966    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy" containerID="cri-o://265b28e58567a1c6de4dd23f7b7d45f4cc657381cf98160982bd3902b03022f4" gracePeriod=30
Jul 28 19:56:03.957646 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:03.957559    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:04.150937 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:04.150885    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:56:04.183124 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:04.183084    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:09.095251 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.095218    2563 generic.go:356] "Generic (PLEG): container finished" podID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerID="265b28e58567a1c6de4dd23f7b7d45f4cc657381cf98160982bd3902b03022f4" exitCode=0
Jul 28 19:56:09.095716 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.095289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" event={"ID":"5e84b594-b5df-4e1a-92d7-a8b1d7b30026","Type":"ContainerDied","Data":"265b28e58567a1c6de4dd23f7b7d45f4cc657381cf98160982bd3902b03022f4"}
Jul 28 19:56:09.095716 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.095357    2563 scope.go:117] "RemoveContainer" containerID="b233f688caadd64305e31b9f7b1c41af0c8e9945f4ffabca65365b543e5667f3"
Jul 28 19:56:09.204712 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.204649    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:56:09.356041 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356009    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-token\") pod \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") "
Jul 28 19:56:09.356041 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356040    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-podinfo\") pod \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") "
Jul 28 19:56:09.356253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356062    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-certs\") pod \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") "
Jul 28 19:56:09.356253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356079    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-envoy\") pod \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") "
Jul 28 19:56:09.356253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356135    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istiod-ca-cert\") pod \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") "
Jul 28 19:56:09.356253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356150    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-credential-socket\") pod \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") "
Jul 28 19:56:09.356253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356166    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb744\" (UniqueName: \"kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-kube-api-access-bb744\") pod \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") "
Jul 28 19:56:09.356505 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356265    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-data\") pod \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") "
Jul 28 19:56:09.356505 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356294    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-socket\") pod \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\" (UID: \"5e84b594-b5df-4e1a-92d7-a8b1d7b30026\") "
Jul 28 19:56:09.356505 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356342    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5e84b594-b5df-4e1a-92d7-a8b1d7b30026" (UID: "5e84b594-b5df-4e1a-92d7-a8b1d7b30026"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:56:09.356505 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356470    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5e84b594-b5df-4e1a-92d7-a8b1d7b30026" (UID: "5e84b594-b5df-4e1a-92d7-a8b1d7b30026"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:56:09.356722 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356562    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5e84b594-b5df-4e1a-92d7-a8b1d7b30026" (UID: "5e84b594-b5df-4e1a-92d7-a8b1d7b30026"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:56:09.356722 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356642    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:09.356722 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356661    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:09.356722 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.356676    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5e84b594-b5df-4e1a-92d7-a8b1d7b30026" (UID: "5e84b594-b5df-4e1a-92d7-a8b1d7b30026"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:56:09.357761 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.357731    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-data" (OuterVolumeSpecName: "istio-data") pod "5e84b594-b5df-4e1a-92d7-a8b1d7b30026" (UID: "5e84b594-b5df-4e1a-92d7-a8b1d7b30026"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:56:09.358480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.358457    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-token" (OuterVolumeSpecName: "istio-token") pod "5e84b594-b5df-4e1a-92d7-a8b1d7b30026" (UID: "5e84b594-b5df-4e1a-92d7-a8b1d7b30026"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:56:09.358480 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.358461    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5e84b594-b5df-4e1a-92d7-a8b1d7b30026" (UID: "5e84b594-b5df-4e1a-92d7-a8b1d7b30026"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:56:09.358585 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.358488    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5e84b594-b5df-4e1a-92d7-a8b1d7b30026" (UID: "5e84b594-b5df-4e1a-92d7-a8b1d7b30026"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:56:09.358775 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.358755    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-kube-api-access-bb744" (OuterVolumeSpecName: "kube-api-access-bb744") pod "5e84b594-b5df-4e1a-92d7-a8b1d7b30026" (UID: "5e84b594-b5df-4e1a-92d7-a8b1d7b30026"). InnerVolumeSpecName "kube-api-access-bb744". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:56:09.457488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.457403    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:09.457488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.457433    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:09.457488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.457443    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:09.457488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.457451    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:09.457488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.457460    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb744\" (UniqueName: \"kubernetes.io/projected/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-kube-api-access-bb744\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:09.457488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.457469    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:09.457488 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:09.457477    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e84b594-b5df-4e1a-92d7-a8b1d7b30026-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:10.100606 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:10.100575    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"
Jul 28 19:56:10.100606 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:10.100597    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4" event={"ID":"5e84b594-b5df-4e1a-92d7-a8b1d7b30026","Type":"ContainerDied","Data":"4b5779b3ccf29a3f9c1ffe8256a72cde78864c2001d333cff31ef44381112ff6"}
Jul 28 19:56:10.101302 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:10.100659    2563 scope.go:117] "RemoveContainer" containerID="265b28e58567a1c6de4dd23f7b7d45f4cc657381cf98160982bd3902b03022f4"
Jul 28 19:56:10.126194 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:10.126160    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"]
Jul 28 19:56:10.131727 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:10.131703    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ec8cba-openshift-default-76bb9f49c6-77gf4"]
Jul 28 19:56:11.143892 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:11.143859    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" path="/var/lib/kubelet/pods/5e84b594-b5df-4e1a-92d7-a8b1d7b30026/volumes"
Jul 28 19:56:22.645116 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:22.645075    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:56:23.424957 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:23.424876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:23.496435 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:23.496373    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:23.565020 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:23.564969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:23.664697 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:23.664667    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:24.079530 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:24.079492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:29.066259 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:29.066167    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"]
Jul 28 19:56:29.159954 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:29.159879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:33.549574 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:33.549543    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:56:44.085546 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.085486    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh" podUID="3844be21-4eb4-4ac7-aa2e-cbbed11c9790" containerName="main" containerID="cri-o://65ab89a10f3ed63fd9207adfbadb3638267c5859ae0d5c14a2125b3836a9f982" gracePeriod=45
Jul 28 19:56:44.254535 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.254506    2563 generic.go:356] "Generic (PLEG): container finished" podID="3844be21-4eb4-4ac7-aa2e-cbbed11c9790" containerID="65ab89a10f3ed63fd9207adfbadb3638267c5859ae0d5c14a2125b3836a9f982" exitCode=0
Jul 28 19:56:44.254651 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.254587    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh" event={"ID":"3844be21-4eb4-4ac7-aa2e-cbbed11c9790","Type":"ContainerDied","Data":"65ab89a10f3ed63fd9207adfbadb3638267c5859ae0d5c14a2125b3836a9f982"}
Jul 28 19:56:44.331935 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.331899    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:56:44.453762 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.453675    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-dshm\") pod \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") "
Jul 28 19:56:44.453762 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.453715    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tmp-dir\") pod \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") "
Jul 28 19:56:44.453762 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.453733    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-home\") pod \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") "
Jul 28 19:56:44.453762 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.453762    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tls-certs\") pod \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") "
Jul 28 19:56:44.454124 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.453801    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-model-cache\") pod \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") "
Jul 28 19:56:44.454124 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.453838    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4s6d\" (UniqueName: \"kubernetes.io/projected/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-kube-api-access-p4s6d\") pod \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\" (UID: \"3844be21-4eb4-4ac7-aa2e-cbbed11c9790\") "
Jul 28 19:56:44.454124 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.454006    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-home" (OuterVolumeSpecName: "home") pod "3844be21-4eb4-4ac7-aa2e-cbbed11c9790" (UID: "3844be21-4eb4-4ac7-aa2e-cbbed11c9790"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:56:44.454124 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.454030    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3844be21-4eb4-4ac7-aa2e-cbbed11c9790" (UID: "3844be21-4eb4-4ac7-aa2e-cbbed11c9790"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:56:44.454264 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.454126    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-model-cache" (OuterVolumeSpecName: "model-cache") pod "3844be21-4eb4-4ac7-aa2e-cbbed11c9790" (UID: "3844be21-4eb4-4ac7-aa2e-cbbed11c9790"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:56:44.454264 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.454155    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tmp-dir\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:44.454264 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.454174    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-home\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:44.456126 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.456106    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-dshm" (OuterVolumeSpecName: "dshm") pod "3844be21-4eb4-4ac7-aa2e-cbbed11c9790" (UID: "3844be21-4eb4-4ac7-aa2e-cbbed11c9790"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:56:44.456201 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.456153    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3844be21-4eb4-4ac7-aa2e-cbbed11c9790" (UID: "3844be21-4eb4-4ac7-aa2e-cbbed11c9790"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:56:44.456245 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.456222    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-kube-api-access-p4s6d" (OuterVolumeSpecName: "kube-api-access-p4s6d") pod "3844be21-4eb4-4ac7-aa2e-cbbed11c9790" (UID: "3844be21-4eb4-4ac7-aa2e-cbbed11c9790"). InnerVolumeSpecName "kube-api-access-p4s6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:56:44.554982 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.554948    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-dshm\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:44.554982 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.554982    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-tls-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:44.555241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.554992    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-model-cache\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:44.555241 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:44.555002    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4s6d\" (UniqueName: \"kubernetes.io/projected/3844be21-4eb4-4ac7-aa2e-cbbed11c9790-kube-api-access-p4s6d\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:56:45.259726 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:45.259690    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"
Jul 28 19:56:45.259726 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:45.259694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh" event={"ID":"3844be21-4eb4-4ac7-aa2e-cbbed11c9790","Type":"ContainerDied","Data":"b49973e9c56537faf2812f225b26a576a00056e8130eca749620e56ff80442ff"}
Jul 28 19:56:45.260302 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:45.259745    2563 scope.go:117] "RemoveContainer" containerID="65ab89a10f3ed63fd9207adfbadb3638267c5859ae0d5c14a2125b3836a9f982"
Jul 28 19:56:45.275157 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:45.275131    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"]
Jul 28 19:56:45.280029 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:45.280006    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-cf1393/ipp-route-cf1393-kserve-6b7495c68c-d26dh"]
Jul 28 19:56:47.144318 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:56:47.144282    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3844be21-4eb4-4ac7-aa2e-cbbed11c9790" path="/var/lib/kubelet/pods/3844be21-4eb4-4ac7-aa2e-cbbed11c9790/volumes"
Jul 28 19:57:06.235337 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:06.235302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:07.424022 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:07.423985    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:12.169080 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.169041    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:12.266803 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.266764    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:12.428597 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.428517    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"]
Jul 28 19:57:12.428747 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.428727    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy" containerID="cri-o://bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84" gracePeriod=30
Jul 28 19:57:12.447481 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.447431    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:12.781128 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.781031    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:12.885623 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.885584    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"]
Jul 28 19:57:12.886533 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.886507    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3844be21-4eb4-4ac7-aa2e-cbbed11c9790" containerName="main"
Jul 28 19:57:12.886700 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.886688    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3844be21-4eb4-4ac7-aa2e-cbbed11c9790" containerName="main"
Jul 28 19:57:12.886822 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.886812    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy"
Jul 28 19:57:12.886898 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.886889    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy"
Jul 28 19:57:12.887022 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.887013    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy"
Jul 28 19:57:12.887094 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.887086    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy"
Jul 28 19:57:12.887378 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.887318    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy"
Jul 28 19:57:12.887496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.887411    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e84b594-b5df-4e1a-92d7-a8b1d7b30026" containerName="istio-proxy"
Jul 28 19:57:12.887602 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.887589    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3844be21-4eb4-4ac7-aa2e-cbbed11c9790" containerName="main"
Jul 28 19:57:12.890761 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.890743    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"
Jul 28 19:57:12.894108 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.893742    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:57:12.894108 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.893751    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:57:12.894108 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:12.893794    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"]
Jul 28 19:57:13.006949 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:13.006892    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63751134-cf7b-4394-824f-a11f9b97e573-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54\" (UID: \"63751134-cf7b-4394-824f-a11f9b97e573\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"
Jul 28 19:57:13.108253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:13.108220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63751134-cf7b-4394-824f-a11f9b97e573-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54\" (UID: \"63751134-cf7b-4394-824f-a11f9b97e573\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"
Jul 28 19:57:13.108850 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:13.108829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63751134-cf7b-4394-824f-a11f9b97e573-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54\" (UID: \"63751134-cf7b-4394-824f-a11f9b97e573\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"
Jul 28 19:57:13.204400 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:13.204362    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"
Jul 28 19:57:13.337312 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:13.337288    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"]
Jul 28 19:57:13.339812 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:57:13.339787    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63751134_cf7b_4394_824f_a11f9b97e573.slice/crio-51dec78121bf5a01c14fee134b1d1a6720d7ed49f6e55af35adec2850d3c22e8 WatchSource:0}: Error finding container 51dec78121bf5a01c14fee134b1d1a6720d7ed49f6e55af35adec2850d3c22e8: Status 404 returned error can't find the container with id 51dec78121bf5a01c14fee134b1d1a6720d7ed49f6e55af35adec2850d3c22e8
Jul 28 19:57:13.388733 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:13.388701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54" event={"ID":"63751134-cf7b-4394-824f-a11f9b97e573","Type":"ContainerStarted","Data":"51dec78121bf5a01c14fee134b1d1a6720d7ed49f6e55af35adec2850d3c22e8"}
Jul 28 19:57:14.395226 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:14.395191    2563 generic.go:356] "Generic (PLEG): container finished" podID="63751134-cf7b-4394-824f-a11f9b97e573" containerID="bf95b53e8fefaa645c0a8b572925d5aeb7d69cc25a648f0f7ef295e87fad15ea" exitCode=0
Jul 28 19:57:14.395629 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:14.395279    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54" event={"ID":"63751134-cf7b-4394-824f-a11f9b97e573","Type":"ContainerDied","Data":"bf95b53e8fefaa645c0a8b572925d5aeb7d69cc25a648f0f7ef295e87fad15ea"}
Jul 28 19:57:15.531319 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:15.531291    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"
Jul 28 19:57:15.630189 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:15.630154    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63751134-cf7b-4394-824f-a11f9b97e573-maas-api-service-ca\") pod \"63751134-cf7b-4394-824f-a11f9b97e573\" (UID: \"63751134-cf7b-4394-824f-a11f9b97e573\") "
Jul 28 19:57:15.630541 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:15.630516    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63751134-cf7b-4394-824f-a11f9b97e573-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "63751134-cf7b-4394-824f-a11f9b97e573" (UID: "63751134-cf7b-4394-824f-a11f9b97e573"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:15.731151 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:15.731062    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63751134-cf7b-4394-824f-a11f9b97e573-maas-api-service-ca\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:16.406763 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:16.406732    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54" event={"ID":"63751134-cf7b-4394-824f-a11f9b97e573","Type":"ContainerDied","Data":"51dec78121bf5a01c14fee134b1d1a6720d7ed49f6e55af35adec2850d3c22e8"}
Jul 28 19:57:16.407046 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:16.406777    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51dec78121bf5a01c14fee134b1d1a6720d7ed49f6e55af35adec2850d3c22e8"
Jul 28 19:57:16.407046 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:16.406746    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"
Jul 28 19:57:17.688341 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.688281    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:57:17.748047 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748015    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51e47b57-89ed-4b88-8af4-836785e20af8-istio-podinfo\") pod \"51e47b57-89ed-4b88-8af4-836785e20af8\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") "
Jul 28 19:57:17.748222 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748072    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-istio-token\") pod \"51e47b57-89ed-4b88-8af4-836785e20af8\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") "
Jul 28 19:57:17.748222 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748096    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd4zh\" (UniqueName: \"kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-kube-api-access-bd4zh\") pod \"51e47b57-89ed-4b88-8af4-836785e20af8\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") "
Jul 28 19:57:17.748335 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748288    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-envoy\") pod \"51e47b57-89ed-4b88-8af4-836785e20af8\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") "
Jul 28 19:57:17.748397 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748343    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-socket\") pod \"51e47b57-89ed-4b88-8af4-836785e20af8\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") "
Jul 28 19:57:17.748397 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748377    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-credential-socket\") pod \"51e47b57-89ed-4b88-8af4-836785e20af8\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") "
Jul 28 19:57:17.748496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748414    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-data\") pod \"51e47b57-89ed-4b88-8af4-836785e20af8\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") "
Jul 28 19:57:17.748496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748441    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-certs\") pod \"51e47b57-89ed-4b88-8af4-836785e20af8\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") "
Jul 28 19:57:17.748592 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748494    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51e47b57-89ed-4b88-8af4-836785e20af8-istiod-ca-cert\") pod \"51e47b57-89ed-4b88-8af4-836785e20af8\" (UID: \"51e47b57-89ed-4b88-8af4-836785e20af8\") "
Jul 28 19:57:17.748671 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748645    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "51e47b57-89ed-4b88-8af4-836785e20af8" (UID: "51e47b57-89ed-4b88-8af4-836785e20af8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:17.748900 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748808    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:17.749018 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.748937    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "51e47b57-89ed-4b88-8af4-836785e20af8" (UID: "51e47b57-89ed-4b88-8af4-836785e20af8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:17.749079 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.749018    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51e47b57-89ed-4b88-8af4-836785e20af8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "51e47b57-89ed-4b88-8af4-836785e20af8" (UID: "51e47b57-89ed-4b88-8af4-836785e20af8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:17.749079 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.749023    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "51e47b57-89ed-4b88-8af4-836785e20af8" (UID: "51e47b57-89ed-4b88-8af4-836785e20af8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:17.749754 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.749725    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-data" (OuterVolumeSpecName: "istio-data") pod "51e47b57-89ed-4b88-8af4-836785e20af8" (UID: "51e47b57-89ed-4b88-8af4-836785e20af8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:17.750515 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.750489    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-kube-api-access-bd4zh" (OuterVolumeSpecName: "kube-api-access-bd4zh") pod "51e47b57-89ed-4b88-8af4-836785e20af8" (UID: "51e47b57-89ed-4b88-8af4-836785e20af8"). InnerVolumeSpecName "kube-api-access-bd4zh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:17.750971 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.750945    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-istio-token" (OuterVolumeSpecName: "istio-token") pod "51e47b57-89ed-4b88-8af4-836785e20af8" (UID: "51e47b57-89ed-4b88-8af4-836785e20af8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:17.750971 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.750942    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "51e47b57-89ed-4b88-8af4-836785e20af8" (UID: "51e47b57-89ed-4b88-8af4-836785e20af8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:17.751104 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.750974    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51e47b57-89ed-4b88-8af4-836785e20af8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "51e47b57-89ed-4b88-8af4-836785e20af8" (UID: "51e47b57-89ed-4b88-8af4-836785e20af8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:57:17.849474 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.849432    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd4zh\" (UniqueName: \"kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-kube-api-access-bd4zh\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:17.849474 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.849468    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:17.849474 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.849482    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:17.849729 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.849493    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:17.849729 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.849505    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51e47b57-89ed-4b88-8af4-836785e20af8-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:17.849729 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.849516    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51e47b57-89ed-4b88-8af4-836785e20af8-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:17.849729 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.849526    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51e47b57-89ed-4b88-8af4-836785e20af8-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:17.849729 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:17.849535    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51e47b57-89ed-4b88-8af4-836785e20af8-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:18.418713 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.418674    2563 generic.go:356] "Generic (PLEG): container finished" podID="51e47b57-89ed-4b88-8af4-836785e20af8" containerID="bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84" exitCode=0
Jul 28 19:57:18.418942 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.418725    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"
Jul 28 19:57:18.418942 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.418745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" event={"ID":"51e47b57-89ed-4b88-8af4-836785e20af8","Type":"ContainerDied","Data":"bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84"}
Jul 28 19:57:18.418942 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.418798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx" event={"ID":"51e47b57-89ed-4b88-8af4-836785e20af8","Type":"ContainerDied","Data":"4882af6f4b9ad2024539d0e0480b71175cc89b0b084b9fe6d1033be60e023e3a"}
Jul 28 19:57:18.418942 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.418826    2563 scope.go:117] "RemoveContainer" containerID="bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84"
Jul 28 19:57:18.427933 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.427902    2563 scope.go:117] "RemoveContainer" containerID="cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83"
Jul 28 19:57:18.435899 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.435879    2563 scope.go:117] "RemoveContainer" containerID="bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84"
Jul 28 19:57:18.436171 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:57:18.436153    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84\": container with ID starting with bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84 not found: ID does not exist" containerID="bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84"
Jul 28 19:57:18.436220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.436179    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84"} err="failed to get container status \"bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84\": rpc error: code = NotFound desc = could not find container \"bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84\": container with ID starting with bef5985f8ea3cf893f4297b6473e8f4eface9fb733aaeca8d3a264338ef32f84 not found: ID does not exist"
Jul 28 19:57:18.436220 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.436195    2563 scope.go:117] "RemoveContainer" containerID="cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83"
Jul 28 19:57:18.436435 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:57:18.436419    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83\": container with ID starting with cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83 not found: ID does not exist" containerID="cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83"
Jul 28 19:57:18.436492 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.436439    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83"} err="failed to get container status \"cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83\": rpc error: code = NotFound desc = could not find container \"cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83\": container with ID starting with cecc97f42120826be73a58da659178f4e238eeaccefcb9a9c86c68af8dad7d83 not found: ID does not exist"
Jul 28 19:57:18.439533 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.439509    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"]
Jul 28 19:57:18.442795 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:18.442775    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-cf1393-openshift-default-d547d8b9b-hmdsx"]
Jul 28 19:57:19.122624 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:19.122595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:19.146341 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:19.146308    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" path="/var/lib/kubelet/pods/51e47b57-89ed-4b88-8af4-836785e20af8/volumes"
Jul 28 19:57:19.223446 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:19.223417    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:19.299058 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:19.299024    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"]
Jul 28 19:57:19.301737 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:19.301711    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e29fb5-bnp54"]
Jul 28 19:57:19.302640 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:19.302617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:19.387130 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:19.387035    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:19.656110 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:19.656025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:19.974333 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:19.974251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:20.030134 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:20.030105    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:21.143179 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:21.143145    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63751134-cf7b-4394-824f-a11f9b97e573" path="/var/lib/kubelet/pods/63751134-cf7b-4394-824f-a11f9b97e573/volumes"
Jul 28 19:57:22.243187 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:22.243151    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:57:26.380369 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:26.380327    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:26.479029 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:26.478988    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:26.635283 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:26.635205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:26.838683 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:26.838647    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:31.918747 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:31.918713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:31.922708 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:31.922630    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j"]
Jul 28 19:57:31.924491 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:31.924469    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ec81e7-29754465-hzf6j"]
Jul 28 19:57:32.063856 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:32.063828    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:32.118840 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:32.118791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:32.194080 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:32.193971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:32.567441 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:32.567317    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:33.142845 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:33.142814    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8050dd95-8c4c-4ef6-9ca7-098e9450714d" path="/var/lib/kubelet/pods/8050dd95-8c4c-4ef6-9ca7-098e9450714d/volumes"
Jul 28 19:57:37.042274 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:37.042240    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:57:38.985120 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:38.985077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:39.085037 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:39.085002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:39.189539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:39.189512    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:57:39.197025 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:39.196999    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-62f22_02b6ba59-b83b-4614-a80e-4802ed5d66d5/ovn-acl-logging/0.log"
Jul 28 19:57:39.219982 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:39.219949    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"]
Jul 28 19:57:39.220210 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:39.220167    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy" containerID="cri-o://08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c" gracePeriod=30
Jul 28 19:57:39.238215 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:39.238129    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:39.412693 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:39.412658    2563 scope.go:117] "RemoveContainer" containerID="e57ab27a2994ab8d66038e4afe401fa707e8f34a2606d9a00a9b3fe7ee9d00ed"
Jul 28 19:57:39.456486 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:39.456461    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:44.469039 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.469010    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:57:44.535789 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.535683    2563 generic.go:356] "Generic (PLEG): container finished" podID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerID="08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c" exitCode=0
Jul 28 19:57:44.535789 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.535724    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"
Jul 28 19:57:44.535789 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.535763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" event={"ID":"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d","Type":"ContainerDied","Data":"08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c"}
Jul 28 19:57:44.536082 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.535818    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5" event={"ID":"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d","Type":"ContainerDied","Data":"bc3783be19ddd19e1777564e765e5dedd0b90340431e7f2f2eeb1922e241834b"}
Jul 28 19:57:44.536082 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.535846    2563 scope.go:117] "RemoveContainer" containerID="08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c"
Jul 28 19:57:44.544552 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.544532    2563 scope.go:117] "RemoveContainer" containerID="25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25"
Jul 28 19:57:44.552738 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.552718    2563 scope.go:117] "RemoveContainer" containerID="08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c"
Jul 28 19:57:44.553059 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:57:44.553040    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c\": container with ID starting with 08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c not found: ID does not exist" containerID="08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c"
Jul 28 19:57:44.553144 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.553067    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c"} err="failed to get container status \"08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c\": rpc error: code = NotFound desc = could not find container \"08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c\": container with ID starting with 08020cc85df0de16070f13d9b603781827b329e129395681ef4bc3a377a6fc0c not found: ID does not exist"
Jul 28 19:57:44.553144 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.553087    2563 scope.go:117] "RemoveContainer" containerID="25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25"
Jul 28 19:57:44.553344 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:57:44.553326    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25\": container with ID starting with 25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25 not found: ID does not exist" containerID="25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25"
Jul 28 19:57:44.553392 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.553351    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25"} err="failed to get container status \"25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25\": rpc error: code = NotFound desc = could not find container \"25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25\": container with ID starting with 25736b9e2ec5ffb4b69c39d6c4cc6ab00b6772d2b87814af59d6ffe900e79a25 not found: ID does not exist"
Jul 28 19:57:44.588649 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.588613    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-podinfo\") pod \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") "
Jul 28 19:57:44.588776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.588677    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-credential-socket\") pod \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") "
Jul 28 19:57:44.588776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.588705    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-certs\") pod \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") "
Jul 28 19:57:44.588776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.588730    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-token\") pod \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") "
Jul 28 19:57:44.588776 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.588757    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-socket\") pod \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") "
Jul 28 19:57:44.589030 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.588810    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-envoy\") pod \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") "
Jul 28 19:57:44.589030 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.588835    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-data\") pod \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") "
Jul 28 19:57:44.589030 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.588862    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mf5f\" (UniqueName: \"kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-kube-api-access-4mf5f\") pod \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") "
Jul 28 19:57:44.589030 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.588889    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istiod-ca-cert\") pod \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\" (UID: \"5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d\") "
Jul 28 19:57:44.589254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.589032    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" (UID: "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:44.589254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.589048    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" (UID: "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:44.589254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.589220    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:44.589254 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.589240    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:44.589689 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.589268    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" (UID: "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:44.589689 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.589370    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" (UID: "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:44.589689 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.589633    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" (UID: "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:44.591090 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.591066    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" (UID: "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:57:44.591204 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.591087    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" (UID: "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:44.591643 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.591621    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-kube-api-access-4mf5f" (OuterVolumeSpecName: "kube-api-access-4mf5f") pod "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" (UID: "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d"). InnerVolumeSpecName "kube-api-access-4mf5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:44.591718 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.591680    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" (UID: "5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:44.690676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.690633    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:44.690676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.690666    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:44.690676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.690676    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:44.690676 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.690685    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:44.690985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.690693    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:44.690985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.690701    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mf5f\" (UniqueName: \"kubernetes.io/projected/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-kube-api-access-4mf5f\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:44.690985 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.690708    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:44.858243 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.858210    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"]
Jul 28 19:57:44.864173 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:44.864145    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-ec81e7-openshift-default-58c7478496-7l2t5"]
Jul 28 19:57:45.143395 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:45.143295    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" path="/var/lib/kubelet/pods/5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d/volumes"
Jul 28 19:57:47.314431 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:47.314401    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr"]
Jul 28 19:57:47.319154 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:47.319131    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d66111-29754465-nz4zr"]
Jul 28 19:57:48.306253 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:48.306218    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:48.406612 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:48.406577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:49.143593 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:49.143557    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2036a29-7037-45dd-9069-1dfff2f7a542" path="/var/lib/kubelet/pods/e2036a29-7037-45dd-9069-1dfff2f7a542/volumes"
Jul 28 19:57:50.939017 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:50.938975    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rx2dw"]
Jul 28 19:57:53.865550 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:53.865508    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:53.946084 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:53.946054    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:54.095783 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:54.095749    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"]
Jul 28 19:57:54.096058 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:54.096030    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy" containerID="cri-o://747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3" gracePeriod=30
Jul 28 19:57:54.120334 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:54.120252    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:54.378947 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:54.378845    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:59.350062 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.350038    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:57:59.513175 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513091    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-credential-socket\") pod \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") "
Jul 28 19:57:59.513175 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513124    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-socket\") pod \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") "
Jul 28 19:57:59.513175 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513149    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-data\") pod \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") "
Jul 28 19:57:59.513449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513190    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-certs\") pod \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") "
Jul 28 19:57:59.513449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513219    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rtmx\" (UniqueName: \"kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-kube-api-access-8rtmx\") pod \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") "
Jul 28 19:57:59.513449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513236    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-token\") pod \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") "
Jul 28 19:57:59.513449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513287    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-podinfo\") pod \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") "
Jul 28 19:57:59.513449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istiod-ca-cert\") pod \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") "
Jul 28 19:57:59.513449 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513346    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-envoy\") pod \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\" (UID: \"d93cc4c7-a6c2-4583-b871-1686c457ec2f\") "
Jul 28 19:57:59.513749 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513442    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d93cc4c7-a6c2-4583-b871-1686c457ec2f" (UID: "d93cc4c7-a6c2-4583-b871-1686c457ec2f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:59.513749 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513462    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d93cc4c7-a6c2-4583-b871-1686c457ec2f" (UID: "d93cc4c7-a6c2-4583-b871-1686c457ec2f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:59.513749 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513577    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d93cc4c7-a6c2-4583-b871-1686c457ec2f" (UID: "d93cc4c7-a6c2-4583-b871-1686c457ec2f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:59.513749 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513643    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-credential-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:59.513749 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513660    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-socket\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:59.514060 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.513766    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d93cc4c7-a6c2-4583-b871-1686c457ec2f" (UID: "d93cc4c7-a6c2-4583-b871-1686c457ec2f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:59.514154 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.514132    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d93cc4c7-a6c2-4583-b871-1686c457ec2f" (UID: "d93cc4c7-a6c2-4583-b871-1686c457ec2f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:59.515829 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.515803    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d93cc4c7-a6c2-4583-b871-1686c457ec2f" (UID: "d93cc4c7-a6c2-4583-b871-1686c457ec2f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:57:59.515968 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.515801    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d93cc4c7-a6c2-4583-b871-1686c457ec2f" (UID: "d93cc4c7-a6c2-4583-b871-1686c457ec2f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:59.516181 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.516158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d93cc4c7-a6c2-4583-b871-1686c457ec2f" (UID: "d93cc4c7-a6c2-4583-b871-1686c457ec2f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 19:57:59.516261 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.516244    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-kube-api-access-8rtmx" (OuterVolumeSpecName: "kube-api-access-8rtmx") pod "d93cc4c7-a6c2-4583-b871-1686c457ec2f" (UID: "d93cc4c7-a6c2-4583-b871-1686c457ec2f"). InnerVolumeSpecName "kube-api-access-8rtmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:59.603317 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.603281    2563 generic.go:356] "Generic (PLEG): container finished" podID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerID="747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3" exitCode=0
Jul 28 19:57:59.603498 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.603331    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"
Jul 28 19:57:59.603498 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.603353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" event={"ID":"d93cc4c7-a6c2-4583-b871-1686c457ec2f","Type":"ContainerDied","Data":"747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3"}
Jul 28 19:57:59.603498 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.603402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b" event={"ID":"d93cc4c7-a6c2-4583-b871-1686c457ec2f","Type":"ContainerDied","Data":"cab4a9f32ecced2e9b2d681c363924a4b679d9d56b2ab4e5f69b7ff53aa4757a"}
Jul 28 19:57:59.603498 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.603424    2563 scope.go:117] "RemoveContainer" containerID="747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3"
Jul 28 19:57:59.612696 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.612680    2563 scope.go:117] "RemoveContainer" containerID="94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1"
Jul 28 19:57:59.614804 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.614784    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-envoy\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:59.614902 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.614806    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-data\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:59.614902 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.614818    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d93cc4c7-a6c2-4583-b871-1686c457ec2f-workload-certs\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:59.614902 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.614832    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rtmx\" (UniqueName: \"kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-kube-api-access-8rtmx\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:59.614902 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.614844    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-token\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:59.614902 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.614858    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istio-podinfo\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:59.614902 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.614869    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d93cc4c7-a6c2-4583-b871-1686c457ec2f-istiod-ca-cert\") on node \"ip-10-0-138-64.ec2.internal\" DevicePath \"\""
Jul 28 19:57:59.620497 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.620481    2563 scope.go:117] "RemoveContainer" containerID="747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3"
Jul 28 19:57:59.620769 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:57:59.620750    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3\": container with ID starting with 747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3 not found: ID does not exist" containerID="747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3"
Jul 28 19:57:59.620816 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.620778    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3"} err="failed to get container status \"747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3\": rpc error: code = NotFound desc = could not find container \"747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3\": container with ID starting with 747ce952d74a36f0423c08d214008f0467fd532a3acb7d08100cbb0e73407cf3 not found: ID does not exist"
Jul 28 19:57:59.620816 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.620796    2563 scope.go:117] "RemoveContainer" containerID="94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1"
Jul 28 19:57:59.621049 ip-10-0-138-64 kubenswrapper[2563]: E0728 19:57:59.621029    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1\": container with ID starting with 94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1 not found: ID does not exist" containerID="94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1"
Jul 28 19:57:59.621104 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.621056    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1"} err="failed to get container status \"94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1\": rpc error: code = NotFound desc = could not find container \"94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1\": container with ID starting with 94689de4b7da45a083a1469469e7bbb2ddc4bfb4697acb2d2b0a4aeaae4244c1 not found: ID does not exist"
Jul 28 19:57:59.625680 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.625660    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"]
Jul 28 19:57:59.630473 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:57:59.630452    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-d66111-openshift-default-5c45689f8d-9rg4b"]
Jul 28 19:58:00.313769 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:00.313739    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gmf4z_81a7bbf4-6132-442d-ac6b-126910605785/server/0.log"
Jul 28 19:58:00.771032 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:00.771007    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb_0fb6e434-4979-4721-b1bc-f75d05197218/copy-manifests/0.log"
Jul 28 19:58:00.777613 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:00.777584    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb_0fb6e434-4979-4721-b1bc-f75d05197218/manager/0.log"
Jul 28 19:58:01.142657 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:01.142613    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" path="/var/lib/kubelet/pods/d93cc4c7-a6c2-4583-b871-1686c457ec2f/volumes"
Jul 28 19:58:03.453316 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:03.453282    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-rx2dw_313d514b-2f1c-4680-acc2-77d6de5e14a3/limitador/0.log"
Jul 28 19:58:04.035620 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:04.035589    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7qxw8_0d7aaeb7-658a-4bec-8dda-3164f1f87b92/discovery/0.log"
Jul 28 19:58:05.349234 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:05.349202    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6nb89_7ce32be4-d620-4644-8db4-6d96ea24f069/main/0.log"
Jul 28 19:58:11.142432 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:11.142389    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:11.208232 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:11.208196    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:14.198248 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:14.198216    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wl5w9_7c6fda6c-c699-4ce6-a78a-209994e2db79/global-pull-secret-syncer/0.log"
Jul 28 19:58:14.258720 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:14.258681    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log"
Jul 28 19:58:14.375515 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:14.375484    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-64.ec2.internal_20fec549702e81f5927fb23efca53227/haproxy/0.log"
Jul 28 19:58:20.064699 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:20.064673    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-rx2dw_313d514b-2f1c-4680-acc2-77d6de5e14a3/limitador/0.log"
Jul 28 19:58:21.493209 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:21.493177    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:21.574174 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:21.574132    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:21.633619 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:21.633578    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:22.061833 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:22.061804    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-pltw4_141354af-c817-4cf9-b876-6e0e4852afd5/cluster-monitoring-operator/0.log"
Jul 28 19:58:22.375327 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:22.375289    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrrhb_5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0/node-exporter/0.log"
Jul 28 19:58:22.397290 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:22.397262    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrrhb_5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0/kube-rbac-proxy/0.log"
Jul 28 19:58:22.417780 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:22.417754    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrrhb_5d98dbcb-d508-4b66-b2c2-ef2e2cd55eb0/init-textfile/0.log"
Jul 28 19:58:24.716074 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716037    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"]
Jul 28 19:58:24.716539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716435    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy"
Jul 28 19:58:24.716539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716452    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy"
Jul 28 19:58:24.716539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716467    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy"
Jul 28 19:58:24.716539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716476    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy"
Jul 28 19:58:24.716539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716489    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy"
Jul 28 19:58:24.716539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716497    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy"
Jul 28 19:58:24.716539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716508    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy"
Jul 28 19:58:24.716539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716515    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy"
Jul 28 19:58:24.716539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716526    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy"
Jul 28 19:58:24.716539 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716533    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy"
Jul 28 19:58:24.717061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716546    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy"
Jul 28 19:58:24.717061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716554    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy"
Jul 28 19:58:24.717061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716578    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63751134-cf7b-4394-824f-a11f9b97e573" containerName="revoke-keys"
Jul 28 19:58:24.717061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716587    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="63751134-cf7b-4394-824f-a11f9b97e573" containerName="revoke-keys"
Jul 28 19:58:24.717061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716682    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy"
Jul 28 19:58:24.717061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716693    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d93cc4c7-a6c2-4583-b871-1686c457ec2f" containerName="istio-proxy"
Jul 28 19:58:24.717061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716703    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a57e1bf-5b68-4bc0-813f-b33d3ed09c7d" containerName="istio-proxy"
Jul 28 19:58:24.717061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716713    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy"
Jul 28 19:58:24.717061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716722    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="51e47b57-89ed-4b88-8af4-836785e20af8" containerName="istio-proxy"
Jul 28 19:58:24.717061 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.716731    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="63751134-cf7b-4394-824f-a11f9b97e573" containerName="revoke-keys"
Jul 28 19:58:24.719783 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.719765    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.722756 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.722737    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cgg7r\"/\"kube-root-ca.crt\""
Jul 28 19:58:24.722756 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.722750    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cgg7r\"/\"openshift-service-ca.crt\""
Jul 28 19:58:24.723983 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.723967    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cgg7r\"/\"default-dockercfg-jlth6\""
Jul 28 19:58:24.729693 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.729674    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"]
Jul 28 19:58:24.816213 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.816169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-proc\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.816213 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.816217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-podres\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.816476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.816236    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-lib-modules\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.816476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.816328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-sys\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.816476 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.816395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8lgh\" (UniqueName: \"kubernetes.io/projected/b06efc01-8064-42d3-a6e7-6d3d9057a555-kube-api-access-n8lgh\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.917286 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.917248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-lib-modules\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.917286 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.917290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-sys\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.917499 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.917331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8lgh\" (UniqueName: \"kubernetes.io/projected/b06efc01-8064-42d3-a6e7-6d3d9057a555-kube-api-access-n8lgh\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.917499 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.917382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-proc\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.917499 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.917423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-podres\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.917499 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.917428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-sys\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.917499 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.917458    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-lib-modules\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.917653 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.917512    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-proc\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.917653 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.917522    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b06efc01-8064-42d3-a6e7-6d3d9057a555-podres\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:24.925637 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:24.925608    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8lgh\" (UniqueName: \"kubernetes.io/projected/b06efc01-8064-42d3-a6e7-6d3d9057a555-kube-api-access-n8lgh\") pod \"perf-node-gather-daemonset-78q66\" (UID: \"b06efc01-8064-42d3-a6e7-6d3d9057a555\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:25.030153 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:25.030054    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:25.165404 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:25.165289    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"]
Jul 28 19:58:25.168199 ip-10-0-138-64 kubenswrapper[2563]: W0728 19:58:25.168170    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb06efc01_8064_42d3_a6e7_6d3d9057a555.slice/crio-75cd03e90a5d48221b5a60eebcc7cf18b8f42c27ea55d93deb0dfda521bd4e29 WatchSource:0}: Error finding container 75cd03e90a5d48221b5a60eebcc7cf18b8f42c27ea55d93deb0dfda521bd4e29: Status 404 returned error can't find the container with id 75cd03e90a5d48221b5a60eebcc7cf18b8f42c27ea55d93deb0dfda521bd4e29
Jul 28 19:58:25.252897 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:25.252874    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7ddbd6c58c-npmmr_86916ca0-cd77-47fa-b6f1-703e61defc88/console/0.log"
Jul 28 19:58:25.294594 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:25.294572    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-vhsn4_e684e3a5-5494-43a3-884e-655779d70d39/download-server/0.log"
Jul 28 19:58:25.717779 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:25.717744    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66" event={"ID":"b06efc01-8064-42d3-a6e7-6d3d9057a555","Type":"ContainerStarted","Data":"b127419933b6ab9d445ecb840b409688e9382a2ffac68661ce17b7f0396e1cff"}
Jul 28 19:58:25.718226 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:25.717794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66" event={"ID":"b06efc01-8064-42d3-a6e7-6d3d9057a555","Type":"ContainerStarted","Data":"75cd03e90a5d48221b5a60eebcc7cf18b8f42c27ea55d93deb0dfda521bd4e29"}
Jul 28 19:58:25.718226 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:25.717902    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:25.734706 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:25.734658    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66" podStartSLOduration=1.734641761 podStartE2EDuration="1.734641761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:58:25.732778541 +0000 UTC m=+3347.194386244" watchObservedRunningTime="2026-07-28 19:58:25.734641761 +0000 UTC m=+3347.196249464"
Jul 28 19:58:25.826648 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:25.826611    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-6ctm4_163fedcd-2a59-4baf-960b-064da188725e/volume-data-source-validator/0.log"
Jul 28 19:58:26.898372 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:26.898341    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-smkjm_1cf35915-6763-44f1-8d21-5ef270aad4f8/dns/0.log"
Jul 28 19:58:26.917778 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:26.917751    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-smkjm_1cf35915-6763-44f1-8d21-5ef270aad4f8/kube-rbac-proxy/0.log"
Jul 28 19:58:26.961259 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:26.961234    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ckcmv_ed3429b6-f5f2-48ab-a368-28543e3860ad/dns-node-resolver/0.log"
Jul 28 19:58:27.575496 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:27.575467    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9k5kh_645352cb-de5d-4b0b-88dc-670476f0deff/node-ca/0.log"
Jul 28 19:58:28.784037 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:28.783997    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7qxw8_0d7aaeb7-658a-4bec-8dda-3164f1f87b92/discovery/0.log"
Jul 28 19:58:29.817655 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:29.817612    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mj6fw_daa9a9e1-99e7-4bd1-9745-9231f6f83ba5/serve-healthcheck-canary/0.log"
Jul 28 19:58:30.268940 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:30.268870    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-bnnpr_8e7a991a-d67b-44a5-9be4-ad06be8c9d1e/insights-operator/0.log"
Jul 28 19:58:30.270359 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:30.270333    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-bnnpr_8e7a991a-d67b-44a5-9be4-ad06be8c9d1e/insights-operator/1.log"
Jul 28 19:58:30.423834 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:30.423802    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w55h5_0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8/kube-rbac-proxy/0.log"
Jul 28 19:58:30.442582 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:30.442546    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w55h5_0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8/exporter/0.log"
Jul 28 19:58:30.464458 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:30.464433    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w55h5_0e2bd23e-e1a4-4d74-b7ca-065ef5c24fb8/extractor/0.log"
Jul 28 19:58:31.732276 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:31.732251    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-78q66"
Jul 28 19:58:32.946354 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:32.946326    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gmf4z_81a7bbf4-6132-442d-ac6b-126910605785/server/0.log"
Jul 28 19:58:33.234967 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:33.234858    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb_0fb6e434-4979-4721-b1bc-f75d05197218/manager/0.log"
Jul 28 19:58:33.256106 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:33.256082    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-t2nbb_0fb6e434-4979-4721-b1bc-f75d05197218/copy-manifests/0.log"
Jul 28 19:58:34.597478 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:34.597400    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7fbf845658-kdgzf_cf0dae32-852c-417c-8385-46de0f25cd2c/manager/0.log"
Jul 28 19:58:34.861341 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:34.861261    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7qxw8_0d7aaeb7-658a-4bec-8dda-3164f1f87b92/discovery/0.log"
Jul 28 19:58:36.658572 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:36.658542    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7qxw8_0d7aaeb7-658a-4bec-8dda-3164f1f87b92/discovery/0.log"
Jul 28 19:58:38.369035 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:38.369002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:38.470305 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:38.470270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jnj67_a995bb7d-7792-4c63-bbc1-ee1ec70f5986/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:39.438466 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:39.438431    2563 scope.go:117] "RemoveContainer" containerID="697399872f522b27688e7e5f658630da9c52f519165bc67cdec97970f78ffa4a"
Jul 28 19:58:39.603312 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:39.603278    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-6z2zj_e891d093-8beb-46f1-9494-087e1f667296/migrator/0.log"
Jul 28 19:58:39.626875 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:39.626845    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-6z2zj_e891d093-8beb-46f1-9494-087e1f667296/graceful-termination/0.log"
Jul 28 19:58:41.054037 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:41.053950    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8lhnl_b6e8d670-5c7a-4876-8249-85f13a045bab/kube-multus-additional-cni-plugins/0.log"
Jul 28 19:58:41.074094 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:41.074065    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8lhnl_b6e8d670-5c7a-4876-8249-85f13a045bab/egress-router-binary-copy/0.log"
Jul 28 19:58:41.097044 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:41.097003    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8lhnl_b6e8d670-5c7a-4876-8249-85f13a045bab/cni-plugins/0.log"
Jul 28 19:58:41.116469 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:41.116449    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8lhnl_b6e8d670-5c7a-4876-8249-85f13a045bab/bond-cni-plugin/0.log"
Jul 28 19:58:41.135072 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:41.135045    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8lhnl_b6e8d670-5c7a-4876-8249-85f13a045bab/routeoverride-cni/0.log"
Jul 28 19:58:41.154522 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:41.154502    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8lhnl_b6e8d670-5c7a-4876-8249-85f13a045bab/whereabouts-cni-bincopy/0.log"
Jul 28 19:58:41.175895 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:41.175872    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8lhnl_b6e8d670-5c7a-4876-8249-85f13a045bab/whereabouts-cni/0.log"
Jul 28 19:58:41.433673 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:41.433641    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lsrbm_bcb8000e-6bba-4dbc-a735-1022c0c72320/kube-multus/0.log"
Jul 28 19:58:41.541764 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:41.541738    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9xt7v_88ae7e56-364a-4f77-862a-2065daefbcfc/network-metrics-daemon/0.log"
Jul 28 19:58:41.559176 ip-10-0-138-64 kubenswrapper[2563]: I0728 19:58:41.559154    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9xt7v_88ae7e56-364a-4f77-862a-2065daefbcfc/kube-rbac-proxy/0.log"