Jul 28 18:00:48.811108 ip-10-0-141-192 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 18:00:49.192300 ip-10-0-141-192 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 18:00:49.192300 ip-10-0-141-192 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 18:00:49.192300 ip-10-0-141-192 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 18:00:49.192300 ip-10-0-141-192 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 18:00:49.192300 ip-10-0-141-192 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 18:00:49.193387 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.193302    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 18:00:49.196943 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196928    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 18:00:49.196943 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196942    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196947    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196950    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196953    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196956    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196959    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196962    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196964    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196967    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196970    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196972    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196975    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196978    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196980    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196982    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196985    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196988    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196991    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196993    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196995    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 18:00:49.197006 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.196998    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197001    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197004    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197006    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197009    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197011    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197014    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197017    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197020    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197023    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197025    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197029    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197032    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197035    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197037    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197040    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197044    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197046    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197049    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 18:00:49.197493 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197052    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197054    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197058    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197063    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197067    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197070    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197073    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197075    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197078    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197080    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197083    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197085    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197088    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197090    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197093    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197096    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197099    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197102    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197105    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 18:00:49.198000 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197107    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197110    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197114    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197118    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197121    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197124    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197128    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197131    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197133    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197136    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197139    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197142    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197144    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197147    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197149    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197152    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197154    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197157    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197160    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 18:00:49.198464 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197162    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197165    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197167    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197170    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197172    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197175    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197177    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197180    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197183    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197187    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197189    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197192    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197194    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197197    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197200    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197202    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197205    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197207    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197210    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197212    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 18:00:49.198947 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.197215    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 18:00:49.199732 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199719    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 18:00:49.199732 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199730    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199734    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199737    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199740    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199743    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199746    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199749    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199752    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199754    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199757    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199761    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199765    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199768    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199771    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199774    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199777    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199780    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199783    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199786    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 18:00:49.199792 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199788    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199791    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199794    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199797    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199799    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199802    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199805    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199807    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199810    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199824    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199827    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199829    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199832    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199835    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199837    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199840    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199842    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199845    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199847    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 18:00:49.200280 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199850    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199852    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199856    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199861    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199863    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199866    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199869    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199871    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199874    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199876    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199879    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199881    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199884    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199886    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199888    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199892    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199895    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199898    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199901    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 18:00:49.200739 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199903    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199906    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199909    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199911    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199914    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199916    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199919    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199922    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199924    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199927    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199929    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199931    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199934    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199937    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199939    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199942    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199944    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199947    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199950    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199952    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 18:00:49.201244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199955    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199957    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199959    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199963    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199966    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199968    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199971    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199974    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199976    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199979    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199982    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199985    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199987    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199990    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199992    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199995    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.199997    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200000    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200003    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200007    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 18:00:49.201719 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200011    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200447    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200453    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200456    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200459    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200462    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200465    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200468    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200470    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200473    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200476    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200478    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200481    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200484    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200487    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200490    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200493    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200496    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200498    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200502    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 18:00:49.202225 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200506    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200509    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200512    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200515    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200518    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200520    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200523    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200525    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200528    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200531    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200533    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200536    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200538    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200541    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200543    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200546    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200549    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200551    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200554    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 18:00:49.202720 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200556    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200559    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200562    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200564    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200567    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200569    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200572    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200574    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200578    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200580    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200583    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200586    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200588    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200591    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200593    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200596    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200598    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200601    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200603    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200606    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 18:00:49.203204 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200609    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200611    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200613    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200616    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200618    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200621    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200623    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200626    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200628    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200631    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200633    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200635    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200638    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200641    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200643    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200646    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200649    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200654    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200658    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 18:00:49.203697 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200661    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200664    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200667    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200670    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200673    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200675    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200678    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200681    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200683    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200686    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200688    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200691    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200693    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200696    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200699    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200701    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200703    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200706    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200708    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200711    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 18:00:49.204179 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200714    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.200716    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200791    2583 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200798    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200804    2583 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200809    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200824    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200828    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200832    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200836    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200840    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200843    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200846    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200850    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200853    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200856    2583 flags.go:64] FLAG: --cgroup-root=""
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200859    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200862    2583 flags.go:64] FLAG: --client-ca-file=""
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200865    2583 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200868    2583 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200872    2583 flags.go:64] FLAG: --cluster-domain=""
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200875    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200878    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200881    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 18:00:49.204654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200884    2583 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200888    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200891    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200894    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200898    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200901    2583 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200904    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200907    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200910    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200913    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200917    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200920    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200923    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200926    2583 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200929    2583 flags.go:64] FLAG: --enable-server="true"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200932    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200937    2583 flags.go:64] FLAG: --event-burst="100"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200940    2583 flags.go:64] FLAG: --event-qps="50"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200944    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200947    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200949    2583 flags.go:64] FLAG: --eviction-hard=""
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200957    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200960    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200963    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200966    2583 flags.go:64] FLAG: --eviction-soft=""
Jul 28 18:00:49.205248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200969    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200972    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200975    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200978    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200981    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200984    2583 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200986    2583 flags.go:64] FLAG: --feature-gates=""
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200990    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200993    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200996    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.200999    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201002    2583 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201005    2583 flags.go:64] FLAG: --help="false"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201008    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201011    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201013    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201016    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201020    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201023    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201026    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201029    2583 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201031    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201036    2583 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 18:00:49.205849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201038    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201042    2583 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201044    2583 flags.go:64] FLAG: --kube-reserved=""
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201047    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201050    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201054    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201057    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201060    2583 flags.go:64] FLAG: --lock-file=""
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201063    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201066    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201069    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201074    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201077    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201080    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201082    2583 flags.go:64] FLAG: --logging-format="text"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201085    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201088    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201091    2583 flags.go:64] FLAG: --manifest-url=""
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201094    2583 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201098    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201101    2583 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201105    2583 flags.go:64] FLAG: --max-pods="110"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201108    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201111    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201114    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 18:00:49.206445 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201117    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201120    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201123    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201126    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201133    2583 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201136    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201141    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201144    2583 flags.go:64] FLAG: --pod-cidr=""
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201147    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201153    2583 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201156    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201159    2583 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201161    2583 flags.go:64] FLAG: --port="10250"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201166    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201169    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06237770d107a0ede"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201172    2583 flags.go:64] FLAG: --qos-reserved=""
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201175    2583 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201178    2583 flags.go:64] FLAG: --register-node="true"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201180    2583 flags.go:64] FLAG: --register-with-taints=""
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201184    2583 flags.go:64] FLAG: --registry-burst="10"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201187    2583 flags.go:64] FLAG: --registry-qps="5"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201190    2583 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201192    2583 flags.go:64] FLAG: --reserved-memory=""
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201196    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201199    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 18:00:49.207098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201202    2583 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201204    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201207    2583 flags.go:64] FLAG: --runonce="false"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201210    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201213    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201216    2583 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201219    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201221    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201225    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201228    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201231    2583 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201234    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201237    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201241    2583 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201244    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201247    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201250    2583 flags.go:64] FLAG: --system-cgroups=""
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201253    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201258    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201261    2583 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201265    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201270    2583 flags.go:64] FLAG: --tls-min-version=""
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201272    2583 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201275    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201278    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 18:00:49.207720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201281    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201284    2583 flags.go:64] FLAG: --v="2"
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201288    2583 flags.go:64] FLAG: --version="false"
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201292    2583 flags.go:64] FLAG: --vmodule=""
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201295    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201299    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201410    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201414    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201417    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201422    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201425    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201428    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201431    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201433    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201436    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201438    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201441    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201443    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201446    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201449    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201453    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 18:00:49.208635 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201456    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201458    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201461    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201463    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201466    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201469    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201472    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201475    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201478    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201480    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201483    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201486    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201489    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201491    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201494    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201497    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201499    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201502    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201504    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201507    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 18:00:49.209276 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201509    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201512    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201514    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201517    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201519    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201522    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201524    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201527    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201529    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201532    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201538    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201541    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201544    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201546    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201549    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201551    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201554    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201557    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201561    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201564    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 18:00:49.209754 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201566    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201569    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201572    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201574    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201576    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201579    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201582    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201584    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201587    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201589    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201592    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201596    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201599    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201602    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201605    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201608    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201610    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201613    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201616    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 18:00:49.210294 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201618    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201621    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201624    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201626    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201630    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201633    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201635    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201638    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201640    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201643    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201645    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201649    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201652    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201654    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201657    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201659    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201662    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201664    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201667    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201670    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 18:00:49.210746 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201672    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201676    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201679    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201682    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.201685    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.201693    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.209898    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.209913    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.209930    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.209936    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.209997    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210002    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210006    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210009    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 18:00:49.211278 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210012    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210016    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210020    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210024    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210027    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210030    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210033    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210036    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210039    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210041    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210044    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210047    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210051    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210054    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210057    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210060    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210063    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210066    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210069    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 18:00:49.211637 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210072    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210075    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210078    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210081    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210084    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210091    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210094    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210097    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210101    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210105    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210107    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210110    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210113    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210115    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210118    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210120    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210123    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210126    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210129    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 18:00:49.212119 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210132    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210134    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210137    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210140    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210144    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210146    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210149    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210151    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210154    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210156    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210159    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210162    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210164    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210167    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210170    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210172    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210175    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210178    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210181    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210183    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 18:00:49.212578 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210185    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210188    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210190    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210193    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210195    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210198    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210201    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210203    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210206    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210208    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210211    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210214    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210217    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210219    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210222    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210225    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210228    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210232    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210235    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210237    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 18:00:49.213178 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210239    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210242    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210244    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210247    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210249    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210252    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210254    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210257    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210259    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210262    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210264    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210266    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210269    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210271    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210274    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210276    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 18:00:49.213670 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210279    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.210284    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210422    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210427    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210430    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210433    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210436    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210438    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210441    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210444    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210446    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210449    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210451    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210454    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210457    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 18:00:49.214158 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210460    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210462    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210465    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210467    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210470    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210474    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210478    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210480    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210483    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210486    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210489    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210492    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210494    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210497    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210500    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210503    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210505    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210508    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210511    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 18:00:49.214544 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210513    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210516    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210518    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210521    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210523    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210526    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210529    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210531    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210534    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210536    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210539    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210541    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210544    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210547    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210551    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210554    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210556    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210559    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210562    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 18:00:49.215018 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210565    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210567    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210570    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210572    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210577    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210580    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210582    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210585    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210587    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210590    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210593    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210595    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210597    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210600    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210602    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210605    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210607    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210610    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210612    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210617    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 18:00:49.215501 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210620    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210623    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210626    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210629    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210631    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210634    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210637    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210640    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210642    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210645    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210647    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210650    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210653    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210655    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210658    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210660    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210664    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210667    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210669    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210671    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 18:00:49.216038 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210674    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 18:00:49.216546 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210677    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 18:00:49.216546 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210679    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 18:00:49.216546 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210682    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 18:00:49.216546 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210684    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 18:00:49.216546 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210687    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 18:00:49.216546 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210689    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 18:00:49.216546 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:49.210692    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 18:00:49.216546 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.210697    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 18:00:49.216546 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.210827    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 18:00:49.216546 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.215610    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 18:00:49.217181 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.217165    2583 server.go:1019] "Starting client certificate rotation"
Jul 28 18:00:49.217300 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.217282    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 18:00:49.217330 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.217322    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 18:00:49.219794 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.219781    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 18:00:49.225209 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.225194    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 18:00:49.236603 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.236581    2583 log.go:25] "Validated CRI v1 runtime API"
Jul 28 18:00:49.238601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.238588    2583 log.go:25] "Validated CRI v1 image API"
Jul 28 18:00:49.239431 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.239413    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 18:00:49.241828 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.241795    2583 fs.go:135] Filesystem UUIDs: map[0ad9f01b-6d18-428c-a5eb-f125bdfe3e2f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a95fa7ed-5c32-4137-a950-d3720b48ec58:/dev/nvme0n1p4]
Jul 28 18:00:49.241899 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.241837    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 18:00:49.248745 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.248722    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 18:00:49.249931 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.249825    2583 manager.go:216] Machine: {Timestamp:2026-07-28 18:00:49.248076277 +0000 UTC m=+0.324638265 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100229 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23d92182743de1e637214980cc3139 SystemUUID:ec23d921-8274-3de1-e637-214980cc3139 BootID:a4fc7f0b-288c-4597-bffc-db1a297b1992 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:56:9f:41:c3:21 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:56:9f:41:c3:21 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:7d:d4:65:90:8d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 18:00:49.249931 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.249929    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 18:00:49.250057 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.250045    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 18:00:49.252933 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.252910    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 18:00:49.253069 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.252936    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-192.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 18:00:49.253633 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.253623    2583 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 18:00:49.253664 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.253636    2583 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 18:00:49.253664 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.253643    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 18:00:49.254626 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.254616    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 18:00:49.255402 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.255393    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 18:00:49.257046 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.257035    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 18:00:49.257148 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.257139    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 18:00:49.257177 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.257167    2583 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 18:00:49.257209 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.257177    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 18:00:49.257209 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.257190    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 18:00:49.257209 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.257199    2583 kubelet.go:398] "Adding apiserver pod source"
Jul 28 18:00:49.257209 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.257208    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 18:00:49.257647 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.257636    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 18:00:49.257689 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.257657    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 18:00:49.259687 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.259670    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 18:00:49.261597 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.261584    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 18:00:49.261640 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.261614    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 18:00:49.263070 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263056    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 18:00:49.263120 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263077    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 18:00:49.263120 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263086    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 18:00:49.263120 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263094    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 18:00:49.263120 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263103    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 18:00:49.263120 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263110    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 18:00:49.263120 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263118    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 18:00:49.263274 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263124    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 18:00:49.263274 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263130    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 18:00:49.263274 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263136    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 18:00:49.263274 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263149    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 18:00:49.263274 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263155    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 18:00:49.263959 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263947    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 18:00:49.263996 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.263963    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 18:00:49.267165 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.267150    2583 server.go:1268] "Started kubelet"
Jul 28 18:00:49.267331 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.267226    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 18:00:49.267331 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.267259    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 18:00:49.267429 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.267390    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 18:00:49.268092 ip-10-0-141-192 systemd[1]: Started Kubernetes Kubelet.
Jul 28 18:00:49.268966 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.268944    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 18:00:49.269426 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.269414    2583 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 18:00:49.271768 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.271670    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-192.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 18:00:49.271768 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.271702    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 18:00:49.271916 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.271801    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-192.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 18:00:49.275291 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.274864    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 18:00:49.276737 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.275958    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-192.ec2.internal.18c68602e3b1cf1f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-192.ec2.internal,UID:ip-10-0-141-192.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-192.ec2.internal,},FirstTimestamp:2026-07-28 18:00:49.267117855 +0000 UTC m=+0.343679843,LastTimestamp:2026-07-28 18:00:49.267117855 +0000 UTC m=+0.343679843,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-192.ec2.internal,}"
Jul 28 18:00:49.277467 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.277448    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 18:00:49.277551 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.277473    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 18:00:49.277662 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.277640    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:49.278468 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.278161    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 18:00:49.278468 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.278175    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 18:00:49.278468 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.278225    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 18:00:49.278468 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.278319    2583 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 18:00:49.278468 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.278329    2583 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 18:00:49.279228 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.279210    2583 factory.go:55] Registering systemd factory
Jul 28 18:00:49.279228 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.279230    2583 factory.go:223] Registration of the systemd container factory successfully
Jul 28 18:00:49.279498 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.279478    2583 factory.go:153] Registering CRI-O factory
Jul 28 18:00:49.279561 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.279501    2583 factory.go:223] Registration of the crio container factory successfully
Jul 28 18:00:49.279618 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.279573    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 18:00:49.279618 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.279605    2583 factory.go:103] Registering Raw factory
Jul 28 18:00:49.279709 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.279620    2583 manager.go:1196] Started watching for new ooms in manager
Jul 28 18:00:49.280208 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.280186    2583 manager.go:349] Starting recovery of all containers
Jul 28 18:00:49.283353 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.283327    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 18:00:49.283548 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.283503    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-192.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 18:00:49.287706 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.287668    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 18:00:49.292440 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.292421    2583 manager.go:354] Recovery completed
Jul 28 18:00:49.295878 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.295860    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mgbj4"
Jul 28 18:00:49.296678 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.296667    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 18:00:49.298379 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.298365    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 18:00:49.298424 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.298393    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 18:00:49.298424 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.298404    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientPID"
Jul 28 18:00:49.298885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.298872    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 18:00:49.298885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.298883    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 18:00:49.298960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.298890    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 18:00:49.298960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.298897    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 18:00:49.300532 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.300467    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-192.ec2.internal.18c68602e58ed5e4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-192.ec2.internal,UID:ip-10-0-141-192.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-192.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-192.ec2.internal,},FirstTimestamp:2026-07-28 18:00:49.29838026 +0000 UTC m=+0.374942248,LastTimestamp:2026-07-28 18:00:49.29838026 +0000 UTC m=+0.374942248,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-192.ec2.internal,}"
Jul 28 18:00:49.300674 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.300656    2583 policy_none.go:49] "None policy: Start"
Jul 28 18:00:49.300674 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.300675    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 18:00:49.300829 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.300685    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 18:00:49.301790 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.301774    2583 policy_none.go:47] "Start"
Jul 28 18:00:49.302411 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.302394    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 18:00:49.302473 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.302423    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 18:00:49.302473 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.302440    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientPID"
Jul 28 18:00:49.304391 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.304374    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mgbj4"
Jul 28 18:00:49.343211 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.343193    2583 manager.go:337] "Starting Device Plugin manager"
Jul 28 18:00:49.343289 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.343234    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 18:00:49.343289 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.343247    2583 server.go:87] "Starting device plugin registration server"
Jul 28 18:00:49.343553 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.343540    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 18:00:49.343725 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.343555    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 18:00:49.343784 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.343757    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 18:00:49.343860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.343847    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 18:00:49.343917 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.343861    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 18:00:49.344309 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.344280    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 18:00:49.344384 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.344330    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:49.398801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.398778    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 18:00:49.399810 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.398829    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 18:00:49.399810 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.398850    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 18:00:49.399810 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.398880    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 18:00:49.401072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.401057    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 18:00:49.444864 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.444809    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 18:00:49.445655 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.445640    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 18:00:49.445724 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.445670    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 18:00:49.445724 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.445681    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientPID"
Jul 28 18:00:49.445724 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.445710    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.454998 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.454983    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.455059 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.455006    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-192.ec2.internal\": node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:49.486830 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.486791    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:49.499232 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.499210    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal"]
Jul 28 18:00:49.499307 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.499269    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 18:00:49.500547 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.500533    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 18:00:49.500616 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.500565    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 18:00:49.500616 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.500580    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientPID"
Jul 28 18:00:49.502071 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.502057    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 18:00:49.502189 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.502171    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.502228 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.502200    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 18:00:49.502769 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.502754    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 18:00:49.502865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.502776    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 18:00:49.502865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.502789    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 18:00:49.502865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.502799    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientPID"
Jul 28 18:00:49.502865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.502804    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 18:00:49.502865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.502817    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientPID"
Jul 28 18:00:49.504082 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.504067    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.504143 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.504093    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 18:00:49.504768 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.504754    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 18:00:49.504821 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.504793    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 18:00:49.504821 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.504809    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeHasSufficientPID"
Jul 28 18:00:49.528722 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.528701    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-192.ec2.internal\" not found" node="ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.533296 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.533281    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-192.ec2.internal\" not found" node="ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.580397 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.580373    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54337df22542d0d892eb9498fe899d87-config\") pod \"kube-apiserver-proxy-ip-10-0-141-192.ec2.internal\" (UID: \"54337df22542d0d892eb9498fe899d87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.580549 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.580402    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1cd6414dad8eb2fbe4aeb701e7c93699-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal\" (UID: \"1cd6414dad8eb2fbe4aeb701e7c93699\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.580549 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.580428    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cd6414dad8eb2fbe4aeb701e7c93699-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal\" (UID: \"1cd6414dad8eb2fbe4aeb701e7c93699\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.587562 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.587544    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:49.681638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.681591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54337df22542d0d892eb9498fe899d87-config\") pod \"kube-apiserver-proxy-ip-10-0-141-192.ec2.internal\" (UID: \"54337df22542d0d892eb9498fe899d87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.681638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.681646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1cd6414dad8eb2fbe4aeb701e7c93699-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal\" (UID: \"1cd6414dad8eb2fbe4aeb701e7c93699\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.681781 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.681664    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cd6414dad8eb2fbe4aeb701e7c93699-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal\" (UID: \"1cd6414dad8eb2fbe4aeb701e7c93699\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.681781 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.681663    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54337df22542d0d892eb9498fe899d87-config\") pod \"kube-apiserver-proxy-ip-10-0-141-192.ec2.internal\" (UID: \"54337df22542d0d892eb9498fe899d87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.681781 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.681683    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1cd6414dad8eb2fbe4aeb701e7c93699-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal\" (UID: \"1cd6414dad8eb2fbe4aeb701e7c93699\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.681781 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.681689    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cd6414dad8eb2fbe4aeb701e7c93699-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal\" (UID: \"1cd6414dad8eb2fbe4aeb701e7c93699\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.688126 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.688111    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:49.788456 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.788360    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:49.834027 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.833996    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.836877 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:49.836858    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:49.889310 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.889278    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:49.990006 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:49.989971    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:50.090281 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:50.090200    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:50.190946 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:50.190913    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:50.218128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.218089    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 18:00:50.218578 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.218251    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 18:00:50.291117 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:50.291089    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:50.309187 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.309154    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:55:49 +0000 UTC" deadline="2028-04-28 06:39:09.353556108 +0000 UTC"
Jul 28 18:00:50.309187 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.309185    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15348h38m19.044374852s"
Jul 28 18:00:50.392230 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:50.392202    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:50.398434 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:50.398405    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd6414dad8eb2fbe4aeb701e7c93699.slice/crio-e72e1c3512e9d3ca4c5890259474eedabfe093d2322a8fbeaab62b8ebb105a5c WatchSource:0}: Error finding container e72e1c3512e9d3ca4c5890259474eedabfe093d2322a8fbeaab62b8ebb105a5c: Status 404 returned error can't find the container with id e72e1c3512e9d3ca4c5890259474eedabfe093d2322a8fbeaab62b8ebb105a5c
Jul 28 18:00:50.405187 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.405141    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal" event={"ID":"1cd6414dad8eb2fbe4aeb701e7c93699","Type":"ContainerStarted","Data":"e72e1c3512e9d3ca4c5890259474eedabfe093d2322a8fbeaab62b8ebb105a5c"}
Jul 28 18:00:50.407774 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.407757    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:00:50.413073 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:50.413051    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54337df22542d0d892eb9498fe899d87.slice/crio-1be8e71bd1e6a775157a0fe472a8c74b8fee0be193e9b9c32de699edae53a819 WatchSource:0}: Error finding container 1be8e71bd1e6a775157a0fe472a8c74b8fee0be193e9b9c32de699edae53a819: Status 404 returned error can't find the container with id 1be8e71bd1e6a775157a0fe472a8c74b8fee0be193e9b9c32de699edae53a819
Jul 28 18:00:50.453574 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.453544    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 18:00:50.493430 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:50.493400    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:50.593898 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:50.593863    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:50.693998 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:50.693925    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-192.ec2.internal\" not found"
Jul 28 18:00:50.755808 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.755780    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 18:00:50.760905 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.760881    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 18:00:50.778736 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.778716    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:50.788352 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.788333    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 18:00:50.789369 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.789357    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal"
Jul 28 18:00:50.802535 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:50.802510    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 18:00:51.111750 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.111663    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 18:00:51.259863 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.259835    2583 apiserver.go:52] "Watching apiserver"
Jul 28 18:00:51.271573 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.271543    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 18:00:51.272929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.272897    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg","openshift-dns/node-resolver-dfk6x","openshift-image-registry/node-ca-qz6v8","openshift-multus/network-metrics-daemon-4v2hh","openshift-ovn-kubernetes/ovnkube-node-b2tvv","openshift-cluster-node-tuning-operator/tuned-dt786","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal","openshift-multus/multus-additional-cni-plugins-cl2r6","openshift-multus/multus-gt7q9","openshift-network-diagnostics/network-check-target-kbqwj","openshift-network-operator/iptables-alerter-xsjlk","kube-system/global-pull-secret-syncer-tc422","kube-system/konnectivity-agent-f6hcm","kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal"]
Jul 28 18:00:51.274621 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.274594    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:51.274715 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.274675    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:00:51.275931 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.275900    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.277305 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.277276    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.278702 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.278680    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 18:00:51.279618 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.279227    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:51.279618 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.279331    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:00:51.279618 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.279380    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 18:00:51.279618 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.279398    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sv8m7\""
Jul 28 18:00:51.279618 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.279427    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 18:00:51.279618 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.279468    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 18:00:51.280125 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.280106    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7psn4\""
Jul 28 18:00:51.280437 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.280407    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 18:00:51.280530 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.280406    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 18:00:51.280601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.280522    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.281294 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.281148    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 18:00:51.282570 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.282186    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.283478 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.283448    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 18:00:51.283608 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.283527    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.285500 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.285347    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.287559 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.286077    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p76jd\""
Jul 28 18:00:51.287559 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.286698    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 18:00:51.287860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.287841    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 18:00:51.289265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.287830    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 18:00:51.289265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.287986    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 18:00:51.289265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.288035    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 18:00:51.289265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.288115    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 18:00:51.289265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.288275    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 18:00:51.289265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.288503    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 18:00:51.289265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.288741    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zbxrc\""
Jul 28 18:00:51.289265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.288983    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gjvvz\""
Jul 28 18:00:51.289265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.289064    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 18:00:51.289265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.289111    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 18:00:51.289835 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.289289    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 18:00:51.289835 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.289383    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:00:51.292212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-run-openvswitch\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292126    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-etc-selinux\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.292212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292151    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.292399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292155    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52wkc\" (UniqueName: \"kubernetes.io/projected/3c98a16c-de7a-4336-b23c-fcb14ec96265-kube-api-access-52wkc\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:51.292399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292260    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-registration-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.292399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292294    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkff4\" (UniqueName: \"kubernetes.io/projected/6887fd7f-e61b-49b4-8581-a50259ab2be2-kube-api-access-lkff4\") pod \"node-ca-qz6v8\" (UID: \"6887fd7f-e61b-49b4-8581-a50259ab2be2\") " pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.292399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292322    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-log-socket\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292345    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-run-ovn\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292370    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-cni-bin\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292393    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:51.292677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292476    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-run-ovn-kubernetes\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292531    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/40d6f2b0-ca72-482d-89b8-f565795ea504-ovn-node-metrics-cert\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292576    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/40d6f2b0-ca72-482d-89b8-f565795ea504-ovnkube-script-lib\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292621    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2gg7\" (UniqueName: \"kubernetes.io/projected/40d6f2b0-ca72-482d-89b8-f565795ea504-kube-api-access-p2gg7\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292649    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6887fd7f-e61b-49b4-8581-a50259ab2be2-serviceca\") pod \"node-ca-qz6v8\" (UID: \"6887fd7f-e61b-49b4-8581-a50259ab2be2\") " pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.292677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.292974 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-device-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.292974 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292723    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-run-netns\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292974 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292750    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292974 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292786    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-slash\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292974 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292822    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-node-log\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.292974 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.292846    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/40d6f2b0-ca72-482d-89b8-f565795ea504-env-overrides\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.293518 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293218    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 18:00:51.293518 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293487    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs4ln\" (UniqueName: \"kubernetes.io/projected/84dfcd44-dfff-481c-924d-a037e5cedcfe-kube-api-access-xs4ln\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.293636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293522    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-socket-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.293636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-systemd-units\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.293636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293570    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-var-lib-openvswitch\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.293636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293593    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-cni-netd\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293638    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gxjfd\""
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293658    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-sys-fs\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293686    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6887fd7f-e61b-49b4-8581-a50259ab2be2-host\") pod \"node-ca-qz6v8\" (UID: \"6887fd7f-e61b-49b4-8581-a50259ab2be2\") " pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/40d6f2b0-ca72-482d-89b8-f565795ea504-ovnkube-config\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293712    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cskxq\""
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293732    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx4fw\" (UniqueName: \"kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw\") pod \"network-check-target-kbqwj\" (UID: \"f10871a0-d4c5-4c63-8dc7-8f5c5123a523\") " pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293739    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293758    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-kubelet\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293781    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-run-systemd\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293792    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293794    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 18:00:51.293839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.293806    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-etc-openvswitch\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.294509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.294042    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.295382 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.295333    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d68mp\""
Jul 28 18:00:51.295617 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.295601    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 18:00:51.295933 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.295688    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:51.295933 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.295749    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:00:51.296217 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.296197    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 18:00:51.297361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.297339    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 18:00:51.297669 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.297643    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 18:00:51.297915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.297883    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 18:00:51.298759 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.298223    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5pbqg\""
Jul 28 18:00:51.298759 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.298459    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 18:00:51.315408 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.315377    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-z5w9v"
Jul 28 18:00:51.322526 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.322509    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z5w9v"
Jul 28 18:00:51.379710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.379636    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 18:00:51.394292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394260    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ccf66553-5457-4b70-a638-2f0b79c4c5f1-iptables-alerter-script\") pod \"iptables-alerter-xsjlk\" (UID: \"ccf66553-5457-4b70-a638-2f0b79c4c5f1\") " pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.394446 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394301    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-cnibin\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.394446 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394329    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-var-lib-cni-multus\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.394446 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394347    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwbkn\" (UniqueName: \"kubernetes.io/projected/9e09196f-d3fa-4a41-8d0b-73e65d75979e-kube-api-access-fwbkn\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.394446 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394362    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-run\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.394446 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-var-lib-openvswitch\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.394446 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394432    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-cni-netd\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.394679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394450    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-var-lib-openvswitch\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.394679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ccf66553-5457-4b70-a638-2f0b79c4c5f1-host-slash\") pod \"iptables-alerter-xsjlk\" (UID: \"ccf66553-5457-4b70-a638-2f0b79c4c5f1\") " pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.394679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394510    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-cni-netd\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.394679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394542    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-sysconfig\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.394679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394592    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-system-cni-dir\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.394679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/40d6f2b0-ca72-482d-89b8-f565795ea504-ovnkube-config\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.394679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4fw\" (UniqueName: \"kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw\") pod \"network-check-target-kbqwj\" (UID: \"f10871a0-d4c5-4c63-8dc7-8f5c5123a523\") " pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394685    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-kubelet\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-etc-selinux\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394736    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52wkc\" (UniqueName: \"kubernetes.io/projected/3c98a16c-de7a-4336-b23c-fcb14ec96265-kube-api-access-52wkc\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394763    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1d6960ee-cc72-488e-bae6-23608c90ab15-kubelet-config\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394802    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqsm2\" (UniqueName: \"kubernetes.io/projected/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-kube-api-access-vqsm2\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394807    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-kubelet\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394828    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkff4\" (UniqueName: \"kubernetes.io/projected/6887fd7f-e61b-49b4-8581-a50259ab2be2-kube-api-access-lkff4\") pod \"node-ca-qz6v8\" (UID: \"6887fd7f-e61b-49b4-8581-a50259ab2be2\") " pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394871    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-cni-dir\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394909    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-var-lib-cni-bin\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394934    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65b87c8d-03ec-45d8-8d7e-76892c63898c-cni-binary-copy\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.395000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.394959    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/59fb8d62-c741-4255-88c7-bc5d23e182a9-agent-certs\") pod \"konnectivity-agent-f6hcm\" (UID: \"59fb8d62-c741-4255-88c7-bc5d23e182a9\") " pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395003    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-kubernetes\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395038    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-lib-modules\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395062    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-host\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395088    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/65b87c8d-03ec-45d8-8d7e-76892c63898c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395092    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-etc-selinux\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-run-ovn-kubernetes\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/40d6f2b0-ca72-482d-89b8-f565795ea504-ovn-node-metrics-cert\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395222    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/40d6f2b0-ca72-482d-89b8-f565795ea504-ovnkube-config\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395229    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-run-ovn-kubernetes\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395273    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gg7\" (UniqueName: \"kubernetes.io/projected/40d6f2b0-ca72-482d-89b8-f565795ea504-kube-api-access-p2gg7\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.395450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395394    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6887fd7f-e61b-49b4-8581-a50259ab2be2-serviceca\") pod \"node-ca-qz6v8\" (UID: \"6887fd7f-e61b-49b4-8581-a50259ab2be2\") " pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.395943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395523    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 18:00:51.395943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395543    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7vg8\" (UniqueName: \"kubernetes.io/projected/d8a76901-42f5-413e-8fa4-64c94fe7bcf7-kube-api-access-c7vg8\") pod \"node-resolver-dfk6x\" (UID: \"d8a76901-42f5-413e-8fa4-64c94fe7bcf7\") " pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.395943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395586    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.395943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395709    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-run-netns\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.395943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395757    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/59fb8d62-c741-4255-88c7-bc5d23e182a9-konnectivity-ca\") pod \"konnectivity-agent-f6hcm\" (UID: \"59fb8d62-c741-4255-88c7-bc5d23e182a9\") " pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:00:51.395943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395786    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-socket-dir-parent\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.395943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395811    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-sysctl-conf\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.395943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395877    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-sys\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.395943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395906    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-system-cni-dir\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.395943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395938    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-slash\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-node-log\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.395989    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-tuned\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396015    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396025    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6887fd7f-e61b-49b4-8581-a50259ab2be2-serviceca\") pod \"node-ca-qz6v8\" (UID: \"6887fd7f-e61b-49b4-8581-a50259ab2be2\") " pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396085    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/65b87c8d-03ec-45d8-8d7e-76892c63898c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396091    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-node-log\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396115    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n74ph\" (UniqueName: \"kubernetes.io/projected/65b87c8d-03ec-45d8-8d7e-76892c63898c-kube-api-access-n74ph\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396135    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-run-netns\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs4ln\" (UniqueName: \"kubernetes.io/projected/84dfcd44-dfff-481c-924d-a037e5cedcfe-kube-api-access-xs4ln\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396174    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8a76901-42f5-413e-8fa4-64c94fe7bcf7-hosts-file\") pod \"node-resolver-dfk6x\" (UID: \"d8a76901-42f5-413e-8fa4-64c94fe7bcf7\") " pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396181    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-run-k8s-cni-cncf-io\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396229    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-socket-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396266    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-systemd-units\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.396349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396296    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-slash\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396359    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-socket-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396394    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-sys-fs\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396419    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6887fd7f-e61b-49b4-8581-a50259ab2be2-host\") pod \"node-ca-qz6v8\" (UID: \"6887fd7f-e61b-49b4-8581-a50259ab2be2\") " pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396455    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-cnibin\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396479    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e09196f-d3fa-4a41-8d0b-73e65d75979e-cni-binary-copy\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396501    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-tmp\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396539    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-os-release\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396567    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-run-systemd\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396585    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6887fd7f-e61b-49b4-8581-a50259ab2be2-host\") pod \"node-ca-qz6v8\" (UID: \"6887fd7f-e61b-49b4-8581-a50259ab2be2\") " pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-etc-openvswitch\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396622    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-run-openvswitch\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396624    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-systemd-units\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396667    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-run-systemd\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396670    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-modprobe-d\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396697    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-var-lib-kubelet\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396736    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-os-release\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.397017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396763    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-registration-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396733    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-sys-fs\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396807    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-daemon-config\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396808    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-run-openvswitch\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396837    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-etc-openvswitch\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-registration-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-log-socket\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396964    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-log-socket\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.396989    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-run-ovn\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397006    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-cni-bin\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397045    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397051    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-cni-bin\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397075    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-etc-kubernetes\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397095    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-run-ovn\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397178    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27wg7\" (UniqueName: \"kubernetes.io/projected/ccf66553-5457-4b70-a638-2f0b79c4c5f1-kube-api-access-27wg7\") pod \"iptables-alerter-xsjlk\" (UID: \"ccf66553-5457-4b70-a638-2f0b79c4c5f1\") " pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.397184    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1d6960ee-cc72-488e-bae6-23608c90ab15-dbus\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:51.397853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397233    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-sysctl-d\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.397283    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs podName:3c98a16c-de7a-4336-b23c-fcb14ec96265 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:51.897247462 +0000 UTC m=+2.973809460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs") pod "network-metrics-daemon-4v2hh" (UID: "3c98a16c-de7a-4336-b23c-fcb14ec96265") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/40d6f2b0-ca72-482d-89b8-f565795ea504-ovnkube-script-lib\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397328    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-systemd\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397388    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-device-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397457    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397459    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84dfcd44-dfff-481c-924d-a037e5cedcfe-device-dir\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397504    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-run-netns\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397511    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40d6f2b0-ca72-482d-89b8-f565795ea504-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397545    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-var-lib-kubelet\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397571    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-hostroot\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397602    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-run-multus-certs\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397636    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8a76901-42f5-413e-8fa4-64c94fe7bcf7-tmp-dir\") pod \"node-resolver-dfk6x\" (UID: \"d8a76901-42f5-413e-8fa4-64c94fe7bcf7\") " pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397667    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/40d6f2b0-ca72-482d-89b8-f565795ea504-env-overrides\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397711    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-conf-dir\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.397877    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/40d6f2b0-ca72-482d-89b8-f565795ea504-ovnkube-script-lib\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.398734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.398539    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/40d6f2b0-ca72-482d-89b8-f565795ea504-env-overrides\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.399961 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.399937    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/40d6f2b0-ca72-482d-89b8-f565795ea504-ovn-node-metrics-cert\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.406987 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.405208    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 18:00:51.406987 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.405239    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 18:00:51.406987 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.405255    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vx4fw for pod openshift-network-diagnostics/network-check-target-kbqwj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:00:51.406987 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.405329    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw podName:f10871a0-d4c5-4c63-8dc7-8f5c5123a523 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:51.905309198 +0000 UTC m=+2.981871196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vx4fw" (UniqueName: "kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw") pod "network-check-target-kbqwj" (UID: "f10871a0-d4c5-4c63-8dc7-8f5c5123a523") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:00:51.407297 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.407224    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs4ln\" (UniqueName: \"kubernetes.io/projected/84dfcd44-dfff-481c-924d-a037e5cedcfe-kube-api-access-xs4ln\") pod \"aws-ebs-csi-driver-node-ksvcg\" (UID: \"84dfcd44-dfff-481c-924d-a037e5cedcfe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.408242 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.408106    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gg7\" (UniqueName: \"kubernetes.io/projected/40d6f2b0-ca72-482d-89b8-f565795ea504-kube-api-access-p2gg7\") pod \"ovnkube-node-b2tvv\" (UID: \"40d6f2b0-ca72-482d-89b8-f565795ea504\") " pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.409471 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.409444    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52wkc\" (UniqueName: \"kubernetes.io/projected/3c98a16c-de7a-4336-b23c-fcb14ec96265-kube-api-access-52wkc\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:51.409796 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.409760    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal" event={"ID":"54337df22542d0d892eb9498fe899d87","Type":"ContainerStarted","Data":"1be8e71bd1e6a775157a0fe472a8c74b8fee0be193e9b9c32de699edae53a819"}
Jul 28 18:00:51.410026 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.410007    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkff4\" (UniqueName: \"kubernetes.io/projected/6887fd7f-e61b-49b4-8581-a50259ab2be2-kube-api-access-lkff4\") pod \"node-ca-qz6v8\" (UID: \"6887fd7f-e61b-49b4-8581-a50259ab2be2\") " pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.498369 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-etc-kubernetes\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498369    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27wg7\" (UniqueName: \"kubernetes.io/projected/ccf66553-5457-4b70-a638-2f0b79c4c5f1-kube-api-access-27wg7\") pod \"iptables-alerter-xsjlk\" (UID: \"ccf66553-5457-4b70-a638-2f0b79c4c5f1\") " pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.498562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498412    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1d6960ee-cc72-488e-bae6-23608c90ab15-dbus\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:51.498562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498438    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-sysctl-d\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.498562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498463    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-systemd\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.498562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498485    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-run-netns\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498486    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-etc-kubernetes\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498543    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-systemd\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498599    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-run-netns\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498636    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-sysctl-d\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498644    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1d6960ee-cc72-488e-bae6-23608c90ab15-dbus\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498654    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-var-lib-kubelet\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498693    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-hostroot\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498699    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-var-lib-kubelet\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498721    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-run-multus-certs\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498747    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-hostroot\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498751    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8a76901-42f5-413e-8fa4-64c94fe7bcf7-tmp-dir\") pod \"node-resolver-dfk6x\" (UID: \"d8a76901-42f5-413e-8fa4-64c94fe7bcf7\") " pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-conf-dir\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498786    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-run-multus-certs\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498798    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ccf66553-5457-4b70-a638-2f0b79c4c5f1-iptables-alerter-script\") pod \"iptables-alerter-xsjlk\" (UID: \"ccf66553-5457-4b70-a638-2f0b79c4c5f1\") " pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-conf-dir\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.498935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498909    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-cnibin\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498952    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-var-lib-cni-multus\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.498979    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwbkn\" (UniqueName: \"kubernetes.io/projected/9e09196f-d3fa-4a41-8d0b-73e65d75979e-kube-api-access-fwbkn\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499009    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-run\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499008    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-cnibin\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499019    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-var-lib-cni-multus\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ccf66553-5457-4b70-a638-2f0b79c4c5f1-host-slash\") pod \"iptables-alerter-xsjlk\" (UID: \"ccf66553-5457-4b70-a638-2f0b79c4c5f1\") " pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499074    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-run\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499084    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-sysconfig\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499110    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-system-cni-dir\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499119    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ccf66553-5457-4b70-a638-2f0b79c4c5f1-host-slash\") pod \"iptables-alerter-xsjlk\" (UID: \"ccf66553-5457-4b70-a638-2f0b79c4c5f1\") " pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499135    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8a76901-42f5-413e-8fa4-64c94fe7bcf7-tmp-dir\") pod \"node-resolver-dfk6x\" (UID: \"d8a76901-42f5-413e-8fa4-64c94fe7bcf7\") " pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499175    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-system-cni-dir\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499189    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1d6960ee-cc72-488e-bae6-23608c90ab15-kubelet-config\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499201    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-sysconfig\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499159    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1d6960ee-cc72-488e-bae6-23608c90ab15-kubelet-config\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499261    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqsm2\" (UniqueName: \"kubernetes.io/projected/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-kube-api-access-vqsm2\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499291    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-cni-dir\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.499601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499320    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-var-lib-cni-bin\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499349    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65b87c8d-03ec-45d8-8d7e-76892c63898c-cni-binary-copy\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499374    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/59fb8d62-c741-4255-88c7-bc5d23e182a9-agent-certs\") pod \"konnectivity-agent-f6hcm\" (UID: \"59fb8d62-c741-4255-88c7-bc5d23e182a9\") " pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499391    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-cni-dir\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499399    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-kubernetes\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499400    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ccf66553-5457-4b70-a638-2f0b79c4c5f1-iptables-alerter-script\") pod \"iptables-alerter-xsjlk\" (UID: \"ccf66553-5457-4b70-a638-2f0b79c4c5f1\") " pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499435    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-lib-modules\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499453    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-host\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499455    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-var-lib-cni-bin\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499471    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/65b87c8d-03ec-45d8-8d7e-76892c63898c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499461    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-kubernetes\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499517    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-host\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499564    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7vg8\" (UniqueName: \"kubernetes.io/projected/d8a76901-42f5-413e-8fa4-64c94fe7bcf7-kube-api-access-c7vg8\") pod \"node-resolver-dfk6x\" (UID: \"d8a76901-42f5-413e-8fa4-64c94fe7bcf7\") " pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499589    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-lib-modules\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499595    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/59fb8d62-c741-4255-88c7-bc5d23e182a9-konnectivity-ca\") pod \"konnectivity-agent-f6hcm\" (UID: \"59fb8d62-c741-4255-88c7-bc5d23e182a9\") " pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499640    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-socket-dir-parent\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499667    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-sysctl-conf\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.500540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499693    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-sys\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499721    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-system-cni-dir\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499749    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-tuned\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499802    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/65b87c8d-03ec-45d8-8d7e-76892c63898c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499853    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n74ph\" (UniqueName: \"kubernetes.io/projected/65b87c8d-03ec-45d8-8d7e-76892c63898c-kube-api-access-n74ph\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499896    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8a76901-42f5-413e-8fa4-64c94fe7bcf7-hosts-file\") pod \"node-resolver-dfk6x\" (UID: \"d8a76901-42f5-413e-8fa4-64c94fe7bcf7\") " pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499922    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-run-k8s-cni-cncf-io\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499927    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-system-cni-dir\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499952    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-cnibin\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.499976    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e09196f-d3fa-4a41-8d0b-73e65d75979e-cni-binary-copy\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500001    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-tmp\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500006    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-socket-dir-parent\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500025    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-os-release\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500063    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65b87c8d-03ec-45d8-8d7e-76892c63898c-cni-binary-copy\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500099    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/65b87c8d-03ec-45d8-8d7e-76892c63898c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500147    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-sysctl-conf\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.501514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500177    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-modprobe-d\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-modprobe-d\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500199    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-sys\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500229    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-var-lib-kubelet\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500259    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-os-release\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500287    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-daemon-config\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500347    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500414    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/59fb8d62-c741-4255-88c7-bc5d23e182a9-konnectivity-ca\") pod \"konnectivity-agent-f6hcm\" (UID: \"59fb8d62-c741-4255-88c7-bc5d23e182a9\") " pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.500460    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.500515    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret podName:1d6960ee-cc72-488e-bae6-23608c90ab15 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:52.000497469 +0000 UTC m=+3.077059463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret") pod "global-pull-secret-syncer-tc422" (UID: "1d6960ee-cc72-488e-bae6-23608c90ab15") : object "kube-system"/"original-pull-secret" not registered
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500572    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-var-lib-kubelet\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500196    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-host-run-k8s-cni-cncf-io\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500656    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-os-release\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500727    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-os-release\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500768    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e09196f-d3fa-4a41-8d0b-73e65d75979e-cni-binary-copy\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500780    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e09196f-d3fa-4a41-8d0b-73e65d75979e-cnibin\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500847    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8a76901-42f5-413e-8fa4-64c94fe7bcf7-hosts-file\") pod \"node-resolver-dfk6x\" (UID: \"d8a76901-42f5-413e-8fa4-64c94fe7bcf7\") " pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.502355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.500922    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65b87c8d-03ec-45d8-8d7e-76892c63898c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.503023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.501117    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e09196f-d3fa-4a41-8d0b-73e65d75979e-multus-daemon-config\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.503023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.501260    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/65b87c8d-03ec-45d8-8d7e-76892c63898c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.503023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.502432    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-etc-tuned\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.503023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.502662    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/59fb8d62-c741-4255-88c7-bc5d23e182a9-agent-certs\") pod \"konnectivity-agent-f6hcm\" (UID: \"59fb8d62-c741-4255-88c7-bc5d23e182a9\") " pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:00:51.503149 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.503118    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-tmp\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.511231 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.511175    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27wg7\" (UniqueName: \"kubernetes.io/projected/ccf66553-5457-4b70-a638-2f0b79c4c5f1-kube-api-access-27wg7\") pod \"iptables-alerter-xsjlk\" (UID: \"ccf66553-5457-4b70-a638-2f0b79c4c5f1\") " pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.512190 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.512158    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7vg8\" (UniqueName: \"kubernetes.io/projected/d8a76901-42f5-413e-8fa4-64c94fe7bcf7-kube-api-access-c7vg8\") pod \"node-resolver-dfk6x\" (UID: \"d8a76901-42f5-413e-8fa4-64c94fe7bcf7\") " pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.512295 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.512240    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqsm2\" (UniqueName: \"kubernetes.io/projected/a21642dd-0bfa-4e32-9cc0-85a689ff8bd4-kube-api-access-vqsm2\") pod \"tuned-dt786\" (UID: \"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4\") " pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.513271 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.513250    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwbkn\" (UniqueName: \"kubernetes.io/projected/9e09196f-d3fa-4a41-8d0b-73e65d75979e-kube-api-access-fwbkn\") pod \"multus-gt7q9\" (UID: \"9e09196f-d3fa-4a41-8d0b-73e65d75979e\") " pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.513368 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.513250    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n74ph\" (UniqueName: \"kubernetes.io/projected/65b87c8d-03ec-45d8-8d7e-76892c63898c-kube-api-access-n74ph\") pod \"multus-additional-cni-plugins-cl2r6\" (UID: \"65b87c8d-03ec-45d8-8d7e-76892c63898c\") " pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.597221 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.597171    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg"
Jul 28 18:00:51.607063 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.607040    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qz6v8"
Jul 28 18:00:51.617092 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.617070    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:00:51.624160 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.624140    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dt786"
Jul 28 18:00:51.631346 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.631289    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gt7q9"
Jul 28 18:00:51.639203 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.639179    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cl2r6"
Jul 28 18:00:51.647072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.647044    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:00:51.652752 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.652733    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dfk6x"
Jul 28 18:00:51.659399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.659373    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xsjlk"
Jul 28 18:00:51.903507 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:51.903417    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:51.903690 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.903549    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:00:51.903690 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:51.903619    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs podName:3c98a16c-de7a-4336-b23c-fcb14ec96265 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:52.903601725 +0000 UTC m=+3.980163706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs") pod "network-metrics-daemon-4v2hh" (UID: "3c98a16c-de7a-4336-b23c-fcb14ec96265") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:00:52.004697 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.004658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:52.004921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.004722    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4fw\" (UniqueName: \"kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw\") pod \"network-check-target-kbqwj\" (UID: \"f10871a0-d4c5-4c63-8dc7-8f5c5123a523\") " pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:52.004921 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:52.004884    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 18:00:52.004921 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:52.004906    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 18:00:52.005106 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:52.004932    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 18:00:52.005106 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:52.004946    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vx4fw for pod openshift-network-diagnostics/network-check-target-kbqwj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:00:52.005106 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:52.004967    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret podName:1d6960ee-cc72-488e-bae6-23608c90ab15 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:53.004942046 +0000 UTC m=+4.081504038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret") pod "global-pull-secret-syncer-tc422" (UID: "1d6960ee-cc72-488e-bae6-23608c90ab15") : object "kube-system"/"original-pull-secret" not registered
Jul 28 18:00:52.005106 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:52.004994    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw podName:f10871a0-d4c5-4c63-8dc7-8f5c5123a523 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:53.004981525 +0000 UTC m=+4.081543506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx4fw" (UniqueName: "kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw") pod "network-check-target-kbqwj" (UID: "f10871a0-d4c5-4c63-8dc7-8f5c5123a523") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:00:52.324663 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.324574    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:55:51 +0000 UTC" deadline="2028-05-06 11:30:53.627404824 +0000 UTC"
Jul 28 18:00:52.324663 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.324611    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15545h30m1.302798464s"
Jul 28 18:00:52.375967 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:52.375939    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b87c8d_03ec_45d8_8d7e_76892c63898c.slice/crio-c9adf9719fe29f7e80952a67306ba93964a4518477f9b829e1132c14833f014d WatchSource:0}: Error finding container c9adf9719fe29f7e80952a67306ba93964a4518477f9b829e1132c14833f014d: Status 404 returned error can't find the container with id c9adf9719fe29f7e80952a67306ba93964a4518477f9b829e1132c14833f014d
Jul 28 18:00:52.389924 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:52.389883    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a76901_42f5_413e_8fa4_64c94fe7bcf7.slice/crio-480bf64821ca44d2e8ff9ad96a1191513718966650302f8c64f12777ff78810c WatchSource:0}: Error finding container 480bf64821ca44d2e8ff9ad96a1191513718966650302f8c64f12777ff78810c: Status 404 returned error can't find the container with id 480bf64821ca44d2e8ff9ad96a1191513718966650302f8c64f12777ff78810c
Jul 28 18:00:52.394548 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:52.394521    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda21642dd_0bfa_4e32_9cc0_85a689ff8bd4.slice/crio-a300bf82a64e73a7a619bd1bf061bfaf2acdab6b5397311b15d786b45685e02c WatchSource:0}: Error finding container a300bf82a64e73a7a619bd1bf061bfaf2acdab6b5397311b15d786b45685e02c: Status 404 returned error can't find the container with id a300bf82a64e73a7a619bd1bf061bfaf2acdab6b5397311b15d786b45685e02c
Jul 28 18:00:52.395223 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:52.395197    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e09196f_d3fa_4a41_8d0b_73e65d75979e.slice/crio-0d3698114cf38210a6a929d24e4f6187c146aaa69a1ee441000dff0050ad42df WatchSource:0}: Error finding container 0d3698114cf38210a6a929d24e4f6187c146aaa69a1ee441000dff0050ad42df: Status 404 returned error can't find the container with id 0d3698114cf38210a6a929d24e4f6187c146aaa69a1ee441000dff0050ad42df
Jul 28 18:00:52.396284 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:52.396256    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccf66553_5457_4b70_a638_2f0b79c4c5f1.slice/crio-4f15614d6c3fd33bdc298105fa0469eefc138a3781580e1f1ce46639c2cab24e WatchSource:0}: Error finding container 4f15614d6c3fd33bdc298105fa0469eefc138a3781580e1f1ce46639c2cab24e: Status 404 returned error can't find the container with id 4f15614d6c3fd33bdc298105fa0469eefc138a3781580e1f1ce46639c2cab24e
Jul 28 18:00:52.397473 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:52.397348    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6887fd7f_e61b_49b4_8581_a50259ab2be2.slice/crio-58ebfa93ed9d4240b964bb5e160a89712303331e8092888cc7300477c9ab2871 WatchSource:0}: Error finding container 58ebfa93ed9d4240b964bb5e160a89712303331e8092888cc7300477c9ab2871: Status 404 returned error can't find the container with id 58ebfa93ed9d4240b964bb5e160a89712303331e8092888cc7300477c9ab2871
Jul 28 18:00:52.398605 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:00:52.398555    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40d6f2b0_ca72_482d_89b8_f565795ea504.slice/crio-7e939a70dc10bef05b4f6b3689201d4bfff2636bd83bacc82a4058ff84549439 WatchSource:0}: Error finding container 7e939a70dc10bef05b4f6b3689201d4bfff2636bd83bacc82a4058ff84549439: Status 404 returned error can't find the container with id 7e939a70dc10bef05b4f6b3689201d4bfff2636bd83bacc82a4058ff84549439
Jul 28 18:00:52.412627 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.412599    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dfk6x" event={"ID":"d8a76901-42f5-413e-8fa4-64c94fe7bcf7","Type":"ContainerStarted","Data":"480bf64821ca44d2e8ff9ad96a1191513718966650302f8c64f12777ff78810c"}
Jul 28 18:00:52.414147 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.414089    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f6hcm" event={"ID":"59fb8d62-c741-4255-88c7-bc5d23e182a9","Type":"ContainerStarted","Data":"782a717eeef6c40c68d4e7e33f301ef913fef4406eece0ac49527eb156aa3a92"}
Jul 28 18:00:52.415160 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.415139    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gt7q9" event={"ID":"9e09196f-d3fa-4a41-8d0b-73e65d75979e","Type":"ContainerStarted","Data":"0d3698114cf38210a6a929d24e4f6187c146aaa69a1ee441000dff0050ad42df"}
Jul 28 18:00:52.416502 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.416472    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cl2r6" event={"ID":"65b87c8d-03ec-45d8-8d7e-76892c63898c","Type":"ContainerStarted","Data":"c9adf9719fe29f7e80952a67306ba93964a4518477f9b829e1132c14833f014d"}
Jul 28 18:00:52.417991 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.417968    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg" event={"ID":"84dfcd44-dfff-481c-924d-a037e5cedcfe","Type":"ContainerStarted","Data":"e081596099791dd61aa05a3ccc873ac2311c86133bc93ff7d284582a84cf6734"}
Jul 28 18:00:52.419210 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.419191    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" event={"ID":"40d6f2b0-ca72-482d-89b8-f565795ea504","Type":"ContainerStarted","Data":"7e939a70dc10bef05b4f6b3689201d4bfff2636bd83bacc82a4058ff84549439"}
Jul 28 18:00:52.420342 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.420320    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qz6v8" event={"ID":"6887fd7f-e61b-49b4-8581-a50259ab2be2","Type":"ContainerStarted","Data":"58ebfa93ed9d4240b964bb5e160a89712303331e8092888cc7300477c9ab2871"}
Jul 28 18:00:52.421439 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.421413    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xsjlk" event={"ID":"ccf66553-5457-4b70-a638-2f0b79c4c5f1","Type":"ContainerStarted","Data":"4f15614d6c3fd33bdc298105fa0469eefc138a3781580e1f1ce46639c2cab24e"}
Jul 28 18:00:52.422659 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.422635    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dt786" event={"ID":"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4","Type":"ContainerStarted","Data":"a300bf82a64e73a7a619bd1bf061bfaf2acdab6b5397311b15d786b45685e02c"}
Jul 28 18:00:52.912721 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:52.912357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:52.912721 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:52.912615    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:00:52.913074 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:52.912783    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs podName:3c98a16c-de7a-4336-b23c-fcb14ec96265 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:54.912758297 +0000 UTC m=+5.989320293 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs") pod "network-metrics-daemon-4v2hh" (UID: "3c98a16c-de7a-4336-b23c-fcb14ec96265") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:00:53.013286 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:53.013241    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4fw\" (UniqueName: \"kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw\") pod \"network-check-target-kbqwj\" (UID: \"f10871a0-d4c5-4c63-8dc7-8f5c5123a523\") " pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:53.013484 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:53.013329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:53.013546 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:53.013481    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 18:00:53.013601 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:53.013554    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret podName:1d6960ee-cc72-488e-bae6-23608c90ab15 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:55.0135337 +0000 UTC m=+6.090095680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret") pod "global-pull-secret-syncer-tc422" (UID: "1d6960ee-cc72-488e-bae6-23608c90ab15") : object "kube-system"/"original-pull-secret" not registered
Jul 28 18:00:53.014031 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:53.014010    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 18:00:53.014131 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:53.014037    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 18:00:53.014131 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:53.014054    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vx4fw for pod openshift-network-diagnostics/network-check-target-kbqwj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:00:53.014131 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:53.014102    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw podName:f10871a0-d4c5-4c63-8dc7-8f5c5123a523 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:55.014084476 +0000 UTC m=+6.090646458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx4fw" (UniqueName: "kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw") pod "network-check-target-kbqwj" (UID: "f10871a0-d4c5-4c63-8dc7-8f5c5123a523") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:00:53.325544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:53.325431    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:55:51 +0000 UTC" deadline="2028-04-19 22:18:39.13920593 +0000 UTC"
Jul 28 18:00:53.325544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:53.325497    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15148h17m45.813713509s"
Jul 28 18:00:53.400685 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:53.399934    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:53.400685 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:53.400061    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:00:53.400685 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:53.400400    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:53.400685 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:53.400505    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:00:53.400685 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:53.400555    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:53.400685 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:53.400631    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:00:53.436840 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:53.436375    2583 generic.go:356] "Generic (PLEG): container finished" podID="1cd6414dad8eb2fbe4aeb701e7c93699" containerID="a7163c32e318fc4d77f23171f83f5acf920e828c340abc2e46db6518a307baf4" exitCode=0
Jul 28 18:00:53.436840 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:53.436476    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal" event={"ID":"1cd6414dad8eb2fbe4aeb701e7c93699","Type":"ContainerDied","Data":"a7163c32e318fc4d77f23171f83f5acf920e828c340abc2e46db6518a307baf4"}
Jul 28 18:00:53.443365 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:53.442956    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal" event={"ID":"54337df22542d0d892eb9498fe899d87","Type":"ContainerStarted","Data":"5c268b31a15e9e9042b424b3355d639af43e8a6185527a5b331b8de77d78a2ed"}
Jul 28 18:00:54.464832 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:54.464791    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal" event={"ID":"1cd6414dad8eb2fbe4aeb701e7c93699","Type":"ContainerStarted","Data":"47b08aabdbeefb49ff3f39028523b32922fcccefdd2a5aff0872ccb0cee0a308"}
Jul 28 18:00:54.481848 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:54.481142    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-192.ec2.internal" podStartSLOduration=4.481121477 podStartE2EDuration="4.481121477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:00:53.471857301 +0000 UTC m=+4.548419301" watchObservedRunningTime="2026-07-28 18:00:54.481121477 +0000 UTC m=+5.557683477"
Jul 28 18:00:54.930901 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:54.930805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:54.931055 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:54.930947    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:00:54.931055 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:54.931013    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs podName:3c98a16c-de7a-4336-b23c-fcb14ec96265 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:58.930993945 +0000 UTC m=+10.007555939 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs") pod "network-metrics-daemon-4v2hh" (UID: "3c98a16c-de7a-4336-b23c-fcb14ec96265") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:00:55.031870 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:55.031825    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:55.032031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:55.031910    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4fw\" (UniqueName: \"kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw\") pod \"network-check-target-kbqwj\" (UID: \"f10871a0-d4c5-4c63-8dc7-8f5c5123a523\") " pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:55.032101 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:55.032064    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 18:00:55.032101 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:55.032081    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 18:00:55.032101 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:55.032095    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vx4fw for pod openshift-network-diagnostics/network-check-target-kbqwj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:00:55.032270 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:55.032151    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw podName:f10871a0-d4c5-4c63-8dc7-8f5c5123a523 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:59.032132907 +0000 UTC m=+10.108694881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx4fw" (UniqueName: "kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw") pod "network-check-target-kbqwj" (UID: "f10871a0-d4c5-4c63-8dc7-8f5c5123a523") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:00:55.032562 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:55.032543    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 18:00:55.032625 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:55.032599    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret podName:1d6960ee-cc72-488e-bae6-23608c90ab15 nodeName:}" failed. No retries permitted until 2026-07-28 18:00:59.032584813 +0000 UTC m=+10.109146791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret") pod "global-pull-secret-syncer-tc422" (UID: "1d6960ee-cc72-488e-bae6-23608c90ab15") : object "kube-system"/"original-pull-secret" not registered
Jul 28 18:00:55.405459 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:55.405382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:55.405601 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:55.405546    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:00:55.406072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:55.406049    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:55.406188 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:55.406167    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:00:55.406274 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:55.406258    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:55.406418 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:55.406398    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:00:57.400101 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:57.400068    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:57.400531 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:57.400204    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:00:57.400611 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:57.400592    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:57.400721 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:57.400700    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:00:57.400932 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:57.400914    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:57.401022 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:57.401004    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:00:58.968743 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:58.968706    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:58.969194 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:58.968898    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:00:58.969194 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:58.968974    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs podName:3c98a16c-de7a-4336-b23c-fcb14ec96265 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:06.968951804 +0000 UTC m=+18.045513789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs") pod "network-metrics-daemon-4v2hh" (UID: "3c98a16c-de7a-4336-b23c-fcb14ec96265") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:00:59.069911 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:59.069872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:59.070097 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:59.069949    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4fw\" (UniqueName: \"kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw\") pod \"network-check-target-kbqwj\" (UID: \"f10871a0-d4c5-4c63-8dc7-8f5c5123a523\") " pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:59.070097 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:59.070086    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 18:00:59.070198 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:59.070102    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 18:00:59.070198 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:59.070114    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vx4fw for pod openshift-network-diagnostics/network-check-target-kbqwj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:00:59.070198 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:59.070166    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw podName:f10871a0-d4c5-4c63-8dc7-8f5c5123a523 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:07.070148949 +0000 UTC m=+18.146710926 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx4fw" (UniqueName: "kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw") pod "network-check-target-kbqwj" (UID: "f10871a0-d4c5-4c63-8dc7-8f5c5123a523") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:00:59.070461 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:59.070437    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 18:00:59.070533 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:59.070494    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret podName:1d6960ee-cc72-488e-bae6-23608c90ab15 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:07.070478909 +0000 UTC m=+18.147040895 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret") pod "global-pull-secret-syncer-tc422" (UID: "1d6960ee-cc72-488e-bae6-23608c90ab15") : object "kube-system"/"original-pull-secret" not registered
Jul 28 18:00:59.400442 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:59.400358    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:00:59.400596 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:59.400469    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:00:59.401134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:59.400852    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:00:59.401134 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:59.400977    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:00:59.401134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:00:59.401023    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:00:59.401134 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:00:59.401100    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:01.399475 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:01.399441    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:01.399475 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:01.399458    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:01.399989 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:01.399571    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:01.399989 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:01.399637    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:01.399989 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:01.399712    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:01.399989 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:01.399797    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:03.399653 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:03.399618    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:03.400135 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:03.399672    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:03.400135 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:03.399750    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:03.400135 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:03.399810    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:03.400135 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:03.399875    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:03.400135 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:03.399927    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:05.400133 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:05.400096    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:05.400656 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:05.400096    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:05.400656 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:05.400257    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:05.400656 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:05.400096    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:05.400656 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:05.400313    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:05.400656 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:05.400391    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:07.029036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:07.029000    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:07.029598 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.029168    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:01:07.029598 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.029242    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs podName:3c98a16c-de7a-4336-b23c-fcb14ec96265 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:23.029225703 +0000 UTC m=+34.105787685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs") pod "network-metrics-daemon-4v2hh" (UID: "3c98a16c-de7a-4336-b23c-fcb14ec96265") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:01:07.130069 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:07.130027    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4fw\" (UniqueName: \"kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw\") pod \"network-check-target-kbqwj\" (UID: \"f10871a0-d4c5-4c63-8dc7-8f5c5123a523\") " pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:07.130245 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:07.130107    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:07.130245 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.130214    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 18:01:07.130245 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.130220    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 18:01:07.130245 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.130245    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 18:01:07.130442 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.130261    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vx4fw for pod openshift-network-diagnostics/network-check-target-kbqwj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:01:07.130442 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.130279    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret podName:1d6960ee-cc72-488e-bae6-23608c90ab15 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:23.130260311 +0000 UTC m=+34.206822288 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret") pod "global-pull-secret-syncer-tc422" (UID: "1d6960ee-cc72-488e-bae6-23608c90ab15") : object "kube-system"/"original-pull-secret" not registered
Jul 28 18:01:07.130442 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.130312    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw podName:f10871a0-d4c5-4c63-8dc7-8f5c5123a523 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:23.130295454 +0000 UTC m=+34.206857449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx4fw" (UniqueName: "kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw") pod "network-check-target-kbqwj" (UID: "f10871a0-d4c5-4c63-8dc7-8f5c5123a523") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:01:07.400020 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:07.399936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:07.400178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:07.399940    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:07.400178 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.400067    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:07.400178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:07.399936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:07.400381 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.400126    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:07.400381 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:07.400245    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:09.400127 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:09.400093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:09.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:09.400189    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:09.400612 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:09.400215    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:09.400612 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:09.400326    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:09.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:09.400334    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:09.400612 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:09.400407    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:11.400271 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.399873    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:11.401030 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.399929    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:11.401030 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:11.400374    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:11.401030 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.399929    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:11.401030 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:11.400456    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:11.401030 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:11.400507    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:11.498655 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.498615    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg" event={"ID":"84dfcd44-dfff-481c-924d-a037e5cedcfe","Type":"ContainerStarted","Data":"9a8213f7a42ce4b678b6475e103ce876738bab2dd744377adb5ecf4428bcc51a"}
Jul 28 18:01:11.501022 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.501001    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" event={"ID":"40d6f2b0-ca72-482d-89b8-f565795ea504","Type":"ContainerStarted","Data":"aa104535de8a9a31922954cb5538bc3a5e36f0974099159e97202172beef8b10"}
Jul 28 18:01:11.501095 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.501061    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" event={"ID":"40d6f2b0-ca72-482d-89b8-f565795ea504","Type":"ContainerStarted","Data":"2764cc07000cc9f63d1fd3f66e5991604344a2b97d9bf5665353830adbd35689"}
Jul 28 18:01:11.501095 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.501093    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" event={"ID":"40d6f2b0-ca72-482d-89b8-f565795ea504","Type":"ContainerStarted","Data":"cea7d5cbce8df8cc11248f86ca110afb748e2b77bd2c3231582f84ab0ad00f64"}
Jul 28 18:01:11.501180 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.501119    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" event={"ID":"40d6f2b0-ca72-482d-89b8-f565795ea504","Type":"ContainerStarted","Data":"5c312bd25799af7187548d8e4c453d515324691d15f1a3e691b8e8747d7f002c"}
Jul 28 18:01:11.501180 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.501153    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" event={"ID":"40d6f2b0-ca72-482d-89b8-f565795ea504","Type":"ContainerStarted","Data":"269bd87ca18459ec7a3f2e7f7e7c72349a88aa2bdb4914962433e19052852920"}
Jul 28 18:01:11.502257 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.502235    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qz6v8" event={"ID":"6887fd7f-e61b-49b4-8581-a50259ab2be2","Type":"ContainerStarted","Data":"c0cc3e0dc8ad5d58faec5cd02511e52f71c078a5af8bd3ed7e89e0f7d370bda2"}
Jul 28 18:01:11.503512 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.503494    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dt786" event={"ID":"a21642dd-0bfa-4e32-9cc0-85a689ff8bd4","Type":"ContainerStarted","Data":"385d8800b4e765129e1207403f2f06a539162005ab916f4bdd0754a913ef273d"}
Jul 28 18:01:11.504664 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.504645    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dfk6x" event={"ID":"d8a76901-42f5-413e-8fa4-64c94fe7bcf7","Type":"ContainerStarted","Data":"3e16495b46c5476dbcd41eb0b1af544da2afc4286153445ea4791e1a8efe6cfa"}
Jul 28 18:01:11.505811 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.505787    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f6hcm" event={"ID":"59fb8d62-c741-4255-88c7-bc5d23e182a9","Type":"ContainerStarted","Data":"2c00873f433a8bf6e0a1070727009d64219d1bf21566faaa8f8700c51ed26fea"}
Jul 28 18:01:11.507058 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.507037    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gt7q9" event={"ID":"9e09196f-d3fa-4a41-8d0b-73e65d75979e","Type":"ContainerStarted","Data":"5a4e8f92b4673659e73c2d206e9463bbe65da98141737c0dbda8406aab35f72b"}
Jul 28 18:01:11.508454 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.508434    2583 generic.go:356] "Generic (PLEG): container finished" podID="65b87c8d-03ec-45d8-8d7e-76892c63898c" containerID="65e582fe1530edb53694545f494281e9bbde56d09672d98b6b887def60bfdb86" exitCode=0
Jul 28 18:01:11.508548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.508461    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cl2r6" event={"ID":"65b87c8d-03ec-45d8-8d7e-76892c63898c","Type":"ContainerDied","Data":"65e582fe1530edb53694545f494281e9bbde56d09672d98b6b887def60bfdb86"}
Jul 28 18:01:11.523228 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.523168    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qz6v8" podStartSLOduration=4.189064818 podStartE2EDuration="22.523147044s" totalImagesPullingTime="18.334082226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:11.519414107 +0000 UTC m=+22.595976109" watchObservedRunningTime="2026-07-28 18:01:11.523147044 +0000 UTC m=+22.599709049"
Jul 28 18:01:11.523901 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.523854    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-192.ec2.internal" podStartSLOduration=21.523835676 podStartE2EDuration="21.523835676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:00:54.482007195 +0000 UTC m=+5.558569193" watchObservedRunningTime="2026-07-28 18:01:11.523835676 +0000 UTC m=+22.600397671"
Jul 28 18:01:11.540586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.540536    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gt7q9" podStartSLOduration=4.14979101 podStartE2EDuration="22.540519102s" totalImagesPullingTime="18.390728092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:11.539673748 +0000 UTC m=+22.616235735" watchObservedRunningTime="2026-07-28 18:01:11.540519102 +0000 UTC m=+22.617081099"
Jul 28 18:01:11.555261 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.555222    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-f6hcm" podStartSLOduration=11.119572955 podStartE2EDuration="21.555205696s" totalImagesPullingTime="10.435632741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:11.555064058 +0000 UTC m=+22.631626052" watchObservedRunningTime="2026-07-28 18:01:11.555205696 +0000 UTC m=+22.631767694"
Jul 28 18:01:11.569425 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.569388    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dfk6x" podStartSLOduration=3.225608153 podStartE2EDuration="21.569377067s" totalImagesPullingTime="18.343768914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:11.569264073 +0000 UTC m=+22.645826071" watchObservedRunningTime="2026-07-28 18:01:11.569377067 +0000 UTC m=+22.645939064"
Jul 28 18:01:11.586749 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:11.586715    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dt786" podStartSLOduration=4.244912613 podStartE2EDuration="22.586704241s" totalImagesPullingTime="18.341791628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:11.586354789 +0000 UTC m=+22.662916789" watchObservedRunningTime="2026-07-28 18:01:11.586704241 +0000 UTC m=+22.663266238"
Jul 28 18:01:12.108771 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:12.108750    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 18:01:12.357089 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:12.356951    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T18:01:12.108766807Z","UUID":"ada20677-3522-42c3-a261-7045093fa5b8","Handler":null,"Name":"","Endpoint":""}
Jul 28 18:01:12.358434 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:12.358419    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 18:01:12.358533 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:12.358451    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 18:01:12.512575 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:12.512537    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg" event={"ID":"84dfcd44-dfff-481c-924d-a037e5cedcfe","Type":"ContainerStarted","Data":"ea51325d8f8286930e6aef3dead26b6375cda840d7eddf0b1bedfd7e9706d3d2"}
Jul 28 18:01:12.516044 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:12.516016    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" event={"ID":"40d6f2b0-ca72-482d-89b8-f565795ea504","Type":"ContainerStarted","Data":"f80ef6a6fe78157a961f50a19c316e586d499e779a1d94d62b080f8117fb59de"}
Jul 28 18:01:12.517670 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:12.517641    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xsjlk" event={"ID":"ccf66553-5457-4b70-a638-2f0b79c4c5f1","Type":"ContainerStarted","Data":"d54613b44cffe2bb5b8666cbfb8e229496c29b386919fdab1c3215b44adb4e7f"}
Jul 28 18:01:13.399681 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:13.399594    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:13.399911 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:13.399594    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:13.399911 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:13.399713    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:13.399911 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:13.399600    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:13.399911 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:13.399865    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:13.400170 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:13.399948    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:13.522059 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:13.522025    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg" event={"ID":"84dfcd44-dfff-481c-924d-a037e5cedcfe","Type":"ContainerStarted","Data":"335e05a440de05ab66a43a817572c75646bff7b6146b101036674377dfb3ee8a"}
Jul 28 18:01:13.541019 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:13.540975    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xsjlk" podStartSLOduration=5.203607302 podStartE2EDuration="23.540954647s" totalImagesPullingTime="18.337347345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:12.536387938 +0000 UTC m=+23.612949935" watchObservedRunningTime="2026-07-28 18:01:13.540954647 +0000 UTC m=+24.617516648"
Jul 28 18:01:14.527432 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:14.527396    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" event={"ID":"40d6f2b0-ca72-482d-89b8-f565795ea504","Type":"ContainerStarted","Data":"7fa79d0c8111b5f9ed0849a50d52c01184d07f4bf63b2575319bacd950b608b3"}
Jul 28 18:01:15.253690 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:15.253494    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:01:15.254121 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:15.254101    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:01:15.271701 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:15.271651    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksvcg" podStartSLOduration=5.935861684 podStartE2EDuration="26.271632309s" totalImagesPullingTime="20.335770625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:13.54206989 +0000 UTC m=+24.618631887" watchObservedRunningTime="2026-07-28 18:01:15.271632309 +0000 UTC m=+26.348194311"
Jul 28 18:01:15.399313 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:15.399284    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:15.399493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:15.399323    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:15.399493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:15.399292    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:15.399493 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:15.399410    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:15.399661 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:15.399493    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:15.399661 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:15.399573    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:16.890360 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:16.890328    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:01:16.890933 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:16.890443    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 18:01:16.891260 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:16.891244    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-f6hcm"
Jul 28 18:01:17.399983 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:17.399767    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:17.400166 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:17.399776    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:17.400166 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:17.400058    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:17.400271 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:17.400176    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:17.400271 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:17.399784    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:17.400338 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:17.400301    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:17.537288 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:17.537249    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" event={"ID":"40d6f2b0-ca72-482d-89b8-f565795ea504","Type":"ContainerStarted","Data":"f37519f40312213d2b58c7b5e8f330f18d2af9513cb4850314193839f0534503"}
Jul 28 18:01:17.537484 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:17.537464    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:01:17.539035 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:17.539012    2583 generic.go:356] "Generic (PLEG): container finished" podID="65b87c8d-03ec-45d8-8d7e-76892c63898c" containerID="2d3b2b3b6910b7b6f967be631f1da317531c1332a2a7f5f18940535fe7795deb" exitCode=0
Jul 28 18:01:17.539151 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:17.539100    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cl2r6" event={"ID":"65b87c8d-03ec-45d8-8d7e-76892c63898c","Type":"ContainerDied","Data":"2d3b2b3b6910b7b6f967be631f1da317531c1332a2a7f5f18940535fe7795deb"}
Jul 28 18:01:17.553828 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:17.553796    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:01:17.567401 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:17.567312    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" podStartSLOduration=10.158183296 podStartE2EDuration="28.567298582s" totalImagesPullingTime="18.409115286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:17.567028022 +0000 UTC m=+28.643590047" watchObservedRunningTime="2026-07-28 18:01:17.567298582 +0000 UTC m=+28.643860578"
Jul 28 18:01:18.541441 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:18.541414    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 18:01:18.541955 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:18.541846    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:01:18.557528 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:18.557509    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:01:18.971082 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:18.970997    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kbqwj"]
Jul 28 18:01:18.971220 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:18.971127    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:18.971220 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:18.971205    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:18.973651 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:18.973630    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4v2hh"]
Jul 28 18:01:18.973755 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:18.973719    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:18.973844 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:18.973809    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:18.976158 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:18.976137    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tc422"]
Jul 28 18:01:18.976256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:18.976210    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:18.976306 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:18.976288    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:19.545789 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:19.545751    2583 generic.go:356] "Generic (PLEG): container finished" podID="65b87c8d-03ec-45d8-8d7e-76892c63898c" containerID="0cdc1d03e59990b8920eb780609175e3fb55d8e3c7829f99eda1f112214c7b55" exitCode=0
Jul 28 18:01:19.546202 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:19.545849    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cl2r6" event={"ID":"65b87c8d-03ec-45d8-8d7e-76892c63898c","Type":"ContainerDied","Data":"0cdc1d03e59990b8920eb780609175e3fb55d8e3c7829f99eda1f112214c7b55"}
Jul 28 18:01:19.546202 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:19.545926    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 18:01:20.371644 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:20.371578    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:01:20.399705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:20.399681    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:20.399838 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:20.399684    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:20.399838 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:20.399774    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:20.399838 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:20.399798    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:20.399970 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:20.399900    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:20.399970 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:20.399958    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:21.565687 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:21.565648    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv" podUID="40d6f2b0-ca72-482d-89b8-f565795ea504" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 28 18:01:22.400045 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:22.400020    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:22.400134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:22.400062    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:22.400134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:22.400020    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:22.400206 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:22.400144    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:22.400274 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:22.400252    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:22.400356 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:22.400339    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:22.555598 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:22.555569    2583 generic.go:356] "Generic (PLEG): container finished" podID="65b87c8d-03ec-45d8-8d7e-76892c63898c" containerID="e7a0f39ed550dcbf05aa109cc938f2ef2a232ce17f2316fcba9d1ceb0638c27a" exitCode=0
Jul 28 18:01:22.555747 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:22.555615    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cl2r6" event={"ID":"65b87c8d-03ec-45d8-8d7e-76892c63898c","Type":"ContainerDied","Data":"e7a0f39ed550dcbf05aa109cc938f2ef2a232ce17f2316fcba9d1ceb0638c27a"}
Jul 28 18:01:23.053521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:23.053491    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:23.053902 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:23.053590    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:01:23.053902 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:23.053638    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs podName:3c98a16c-de7a-4336-b23c-fcb14ec96265 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:55.053623888 +0000 UTC m=+66.130185863 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs") pod "network-metrics-daemon-4v2hh" (UID: "3c98a16c-de7a-4336-b23c-fcb14ec96265") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 18:01:23.154339 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:23.154297    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:23.154501 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:23.154380    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4fw\" (UniqueName: \"kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw\") pod \"network-check-target-kbqwj\" (UID: \"f10871a0-d4c5-4c63-8dc7-8f5c5123a523\") " pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:23.154501 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:23.154474    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 18:01:23.154618 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:23.154541    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 18:01:23.154618 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:23.154557    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 18:01:23.154618 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:23.154567    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vx4fw for pod openshift-network-diagnostics/network-check-target-kbqwj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:01:23.154618 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:23.154557    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret podName:1d6960ee-cc72-488e-bae6-23608c90ab15 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:55.154531525 +0000 UTC m=+66.231093503 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret") pod "global-pull-secret-syncer-tc422" (UID: "1d6960ee-cc72-488e-bae6-23608c90ab15") : object "kube-system"/"original-pull-secret" not registered
Jul 28 18:01:23.154618 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:23.154617    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw podName:f10871a0-d4c5-4c63-8dc7-8f5c5123a523 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:55.154600725 +0000 UTC m=+66.231162700 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx4fw" (UniqueName: "kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw") pod "network-check-target-kbqwj" (UID: "f10871a0-d4c5-4c63-8dc7-8f5c5123a523") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 18:01:24.399806 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.399773    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:24.399806 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.399806    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:24.400350 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.399828    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:24.400350 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:24.399902    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc422" podUID="1d6960ee-cc72-488e-bae6-23608c90ab15"
Jul 28 18:01:24.400350 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:24.399986    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbqwj" podUID="f10871a0-d4c5-4c63-8dc7-8f5c5123a523"
Jul 28 18:01:24.400350 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:24.400084    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v2hh" podUID="3c98a16c-de7a-4336-b23c-fcb14ec96265"
Jul 28 18:01:24.747448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.747411    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-192.ec2.internal" event="NodeReady"
Jul 28 18:01:24.747625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.747583    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 18:01:24.782289 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.782253    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b567bb8ff-8p6pr"]
Jul 28 18:01:24.784288 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.784267    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:24.787508 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.787228    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 18:01:24.787508 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.787294    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8fv8h\""
Jul 28 18:01:24.787508 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.787322    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 18:01:24.787508 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.787463    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 18:01:24.793800 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.793773    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 18:01:24.794351 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.794329    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ftg8z"]
Jul 28 18:01:24.796257 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.796237    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:24.797005 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.796983    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wtkbg"]
Jul 28 18:01:24.799539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.799319    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g9rrw\""
Jul 28 18:01:24.799539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.799325    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 18:01:24.799539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.799345    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 18:01:24.799759 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.799621    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wtkbg"
Jul 28 18:01:24.799759 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.799697    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b567bb8ff-8p6pr"]
Jul 28 18:01:24.802448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.802427    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 18:01:24.803081 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.803058    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 18:01:24.803366 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.803345    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-f89dl\""
Jul 28 18:01:24.803573 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.803535    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 18:01:24.809354 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.809333    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ftg8z"]
Jul 28 18:01:24.812712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.812543    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wtkbg"]
Jul 28 18:01:24.948156 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.948111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:24.966099 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966067    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-installation-pull-secrets\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:24.966281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966111    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-trusted-ca\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:24.966281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966146    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-config-volume\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:24.966281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966181    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-ca-trust-extracted\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:24.966281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966216    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-image-registry-private-configuration\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:24.966281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966258    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:24.966281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966278    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-certificates\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:24.966522 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966337    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:24.966522 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966420    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3fb8585-f057-4f84-9576-14df70802b01-cert\") pod \"ingress-canary-wtkbg\" (UID: \"b3fb8585-f057-4f84-9576-14df70802b01\") " pod="openshift-ingress-canary/ingress-canary-wtkbg"
Jul 28 18:01:24.966522 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966460    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s5nv\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-kube-api-access-7s5nv\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:24.966522 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966486    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-tmp-dir\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:24.966522 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966508    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swqhp\" (UniqueName: \"kubernetes.io/projected/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-kube-api-access-swqhp\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:24.966732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966564    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-bound-sa-token\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:24.966732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:24.966607    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9w9z\" (UniqueName: \"kubernetes.io/projected/b3fb8585-f057-4f84-9576-14df70802b01-kube-api-access-p9w9z\") pod \"ingress-canary-wtkbg\" (UID: \"b3fb8585-f057-4f84-9576-14df70802b01\") " pod="openshift-ingress-canary/ingress-canary-wtkbg"
Jul 28 18:01:25.067128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067098    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-trusted-ca\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.067292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-config-volume\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:25.067292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067169    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-ca-trust-extracted\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.067292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-image-registry-private-configuration\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.067565 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067535    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.067690 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067572    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-certificates\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.067690 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:25.067690 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3fb8585-f057-4f84-9576-14df70802b01-cert\") pod \"ingress-canary-wtkbg\" (UID: \"b3fb8585-f057-4f84-9576-14df70802b01\") " pod="openshift-ingress-canary/ingress-canary-wtkbg"
Jul 28 18:01:25.067690 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s5nv\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-kube-api-access-7s5nv\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.067690 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067643    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-ca-trust-extracted\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.067690 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067667    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-tmp-dir\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:25.067959 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067707    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swqhp\" (UniqueName: \"kubernetes.io/projected/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-kube-api-access-swqhp\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:25.067959 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-bound-sa-token\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.067959 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067765    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9w9z\" (UniqueName: \"kubernetes.io/projected/b3fb8585-f057-4f84-9576-14df70802b01-kube-api-access-p9w9z\") pod \"ingress-canary-wtkbg\" (UID: \"b3fb8585-f057-4f84-9576-14df70802b01\") " pod="openshift-ingress-canary/ingress-canary-wtkbg"
Jul 28 18:01:25.067959 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067840    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-installation-pull-secrets\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.067959 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067893    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-tmp-dir\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:25.068196 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.067977    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-config-volume\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:25.068196 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:25.067993    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 18:01:25.068196 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:25.068008    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b567bb8ff-8p6pr: secret "image-registry-tls" not found
Jul 28 18:01:25.068196 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:25.068055    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls podName:2e61e8b3-e5cd-4e79-ab56-47a81061fdc8 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:25.568039331 +0000 UTC m=+36.644601308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls") pod "image-registry-5b567bb8ff-8p6pr" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8") : secret "image-registry-tls" not found
Jul 28 18:01:25.068382 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.068203    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-trusted-ca\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.068434 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:25.068412    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 18:01:25.068486 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:25.068475    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls podName:fd9c632b-0b01-4a1d-a636-5ce8ec009f9c nodeName:}" failed. No retries permitted until 2026-07-28 18:01:25.568458012 +0000 UTC m=+36.645019991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls") pod "dns-default-ftg8z" (UID: "fd9c632b-0b01-4a1d-a636-5ce8ec009f9c") : secret "dns-default-metrics-tls" not found
Jul 28 18:01:25.068857 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.068789    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-certificates\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.069610 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.069520    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:25.072961 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.072932    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-image-registry-private-configuration\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.073063 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.072957    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3fb8585-f057-4f84-9576-14df70802b01-cert\") pod \"ingress-canary-wtkbg\" (UID: \"b3fb8585-f057-4f84-9576-14df70802b01\") " pod="openshift-ingress-canary/ingress-canary-wtkbg"
Jul 28 18:01:25.073063 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.072970    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-installation-pull-secrets\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.079210 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.079184    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swqhp\" (UniqueName: \"kubernetes.io/projected/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-kube-api-access-swqhp\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:25.079772 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.079748    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9w9z\" (UniqueName: \"kubernetes.io/projected/b3fb8585-f057-4f84-9576-14df70802b01-kube-api-access-p9w9z\") pod \"ingress-canary-wtkbg\" (UID: \"b3fb8585-f057-4f84-9576-14df70802b01\") " pod="openshift-ingress-canary/ingress-canary-wtkbg"
Jul 28 18:01:25.080144 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.080123    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s5nv\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-kube-api-access-7s5nv\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.080673 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.080636    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-bound-sa-token\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.116202 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.116171    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wtkbg"
Jul 28 18:01:25.214325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.214091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:25.270368 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.270275    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wtkbg"]
Jul 28 18:01:25.571083 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.571003    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:25.571083 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:25.571043    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:25.571785 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:25.571152    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 18:01:25.571785 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:25.571169    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b567bb8ff-8p6pr: secret "image-registry-tls" not found
Jul 28 18:01:25.571785 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:25.571193    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 18:01:25.571785 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:25.571235    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls podName:2e61e8b3-e5cd-4e79-ab56-47a81061fdc8 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:26.571215357 +0000 UTC m=+37.647777333 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls") pod "image-registry-5b567bb8ff-8p6pr" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8") : secret "image-registry-tls" not found
Jul 28 18:01:25.571785 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:25.571253    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls podName:fd9c632b-0b01-4a1d-a636-5ce8ec009f9c nodeName:}" failed. No retries permitted until 2026-07-28 18:01:26.571244249 +0000 UTC m=+37.647806224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls") pod "dns-default-ftg8z" (UID: "fd9c632b-0b01-4a1d-a636-5ce8ec009f9c") : secret "dns-default-metrics-tls" not found
Jul 28 18:01:26.399959 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.399928    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:26.400174 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.399971    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:26.400174 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.400078    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:26.404915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.404888    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 18:01:26.405142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.405107    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 18:01:26.405264 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.405166    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x879t\""
Jul 28 18:01:26.405264 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.405177    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9p4qw\""
Jul 28 18:01:26.405264 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.405215    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 18:01:26.405264 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.405224    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 18:01:26.580095 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.580067    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:26.580540 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:26.580186    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:26.580540 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:26.580322    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 18:01:26.580540 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:26.580332    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 18:01:26.580540 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:26.580336    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b567bb8ff-8p6pr: secret "image-registry-tls" not found
Jul 28 18:01:26.580540 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:26.580387    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls podName:2e61e8b3-e5cd-4e79-ab56-47a81061fdc8 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:28.58036924 +0000 UTC m=+39.656931230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls") pod "image-registry-5b567bb8ff-8p6pr" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8") : secret "image-registry-tls" not found
Jul 28 18:01:26.580540 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:26.580400    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls podName:fd9c632b-0b01-4a1d-a636-5ce8ec009f9c nodeName:}" failed. No retries permitted until 2026-07-28 18:01:28.580394295 +0000 UTC m=+39.656956271 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls") pod "dns-default-ftg8z" (UID: "fd9c632b-0b01-4a1d-a636-5ce8ec009f9c") : secret "dns-default-metrics-tls" not found
Jul 28 18:01:26.739313 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:01:26.739279    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3fb8585_f057_4f84_9576_14df70802b01.slice/crio-03dc3e15f044795394b661540b4a9645b048baba013b42a0809feb59e9e7ed74 WatchSource:0}: Error finding container 03dc3e15f044795394b661540b4a9645b048baba013b42a0809feb59e9e7ed74: Status 404 returned error can't find the container with id 03dc3e15f044795394b661540b4a9645b048baba013b42a0809feb59e9e7ed74
Jul 28 18:01:27.463736 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:27.463700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:27.570688 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:27.570656    2583 generic.go:356] "Generic (PLEG): container finished" podID="65b87c8d-03ec-45d8-8d7e-76892c63898c" containerID="8e043876e856f413a2d64d9c763de4b33312a247bca256df123543888107a90a" exitCode=0
Jul 28 18:01:27.570859 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:27.570740    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cl2r6" event={"ID":"65b87c8d-03ec-45d8-8d7e-76892c63898c","Type":"ContainerDied","Data":"8e043876e856f413a2d64d9c763de4b33312a247bca256df123543888107a90a"}
Jul 28 18:01:27.571925 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:27.571874    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wtkbg" event={"ID":"b3fb8585-f057-4f84-9576-14df70802b01","Type":"ContainerStarted","Data":"03dc3e15f044795394b661540b4a9645b048baba013b42a0809feb59e9e7ed74"}
Jul 28 18:01:27.602119 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:27.602096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:27.809133 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:27.809098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:27.934141 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:27.934109    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:28.034416 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:28.034331    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:28.487501 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:28.487465    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:28.558213 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:28.558164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:28.577755 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:28.577724    2583 generic.go:356] "Generic (PLEG): container finished" podID="65b87c8d-03ec-45d8-8d7e-76892c63898c" containerID="74de5e7f5af4d90d80b33780a619852015117ac21ac194ad16d84061d7d77086" exitCode=0
Jul 28 18:01:28.577943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:28.577801    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cl2r6" event={"ID":"65b87c8d-03ec-45d8-8d7e-76892c63898c","Type":"ContainerDied","Data":"74de5e7f5af4d90d80b33780a619852015117ac21ac194ad16d84061d7d77086"}
Jul 28 18:01:28.598667 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:28.598430    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:28.598667 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:28.598471    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:28.598667 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:28.598541    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 18:01:28.598667 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:28.598557    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b567bb8ff-8p6pr: secret "image-registry-tls" not found
Jul 28 18:01:28.598667 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:28.598564    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 18:01:28.598667 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:28.598616    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls podName:fd9c632b-0b01-4a1d-a636-5ce8ec009f9c nodeName:}" failed. No retries permitted until 2026-07-28 18:01:32.598598904 +0000 UTC m=+43.675160892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls") pod "dns-default-ftg8z" (UID: "fd9c632b-0b01-4a1d-a636-5ce8ec009f9c") : secret "dns-default-metrics-tls" not found
Jul 28 18:01:28.598667 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:28.598632    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls podName:2e61e8b3-e5cd-4e79-ab56-47a81061fdc8 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:32.598624111 +0000 UTC m=+43.675186089 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls") pod "image-registry-5b567bb8ff-8p6pr" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8") : secret "image-registry-tls" not found
Jul 28 18:01:29.583515 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:29.583414    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cl2r6" event={"ID":"65b87c8d-03ec-45d8-8d7e-76892c63898c","Type":"ContainerStarted","Data":"dc03c89a47ecc0e46078c13b8bdfbf1ffa8c784b0e38fa7168fc26914e1e6be6"}
Jul 28 18:01:29.586176 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:29.586034    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wtkbg" event={"ID":"b3fb8585-f057-4f84-9576-14df70802b01","Type":"ContainerStarted","Data":"62c11a413706d6fea07e12bece4b2cd845483ce477cd2eb3983ef514277ae519"}
Jul 28 18:01:29.609384 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:29.609339    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cl2r6" podStartSLOduration=9.165369843 podStartE2EDuration="40.609320216s" totalImagesPullingTime="31.443950373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:29.607374965 +0000 UTC m=+40.683936975" watchObservedRunningTime="2026-07-28 18:01:29.609320216 +0000 UTC m=+40.685882214"
Jul 28 18:01:29.622931 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:29.622883    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wtkbg" podStartSLOduration=3.053423923 podStartE2EDuration="5.622867498s" totalImagesPullingTime="2.569443575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:01:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:29.622286105 +0000 UTC m=+40.698848103" watchObservedRunningTime="2026-07-28 18:01:29.622867498 +0000 UTC m=+40.699429495"
Jul 28 18:01:29.738016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:29.737978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:29.837797 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:29.837711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:29.958499 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:29.958453    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:30.205580 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:30.205550    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:30.451762 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:30.451734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:31.482846 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.482796    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7cb56ffbff-lv2ps"]
Jul 28 18:01:31.511266 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.511234    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cb56ffbff-lv2ps"]
Jul 28 18:01:31.511406 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.511338    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.514218 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.514198    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 18:01:31.514567 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.514547    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 18:01:31.514567 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.514553    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 18:01:31.514727 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.514578    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 18:01:31.514727 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.514669    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 18:01:31.514727 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.514701    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-h6vph\""
Jul 28 18:01:31.514904 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.514887    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 18:01:31.576990 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.576957    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv"]
Jul 28 18:01:31.599275 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.599247    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv"]
Jul 28 18:01:31.599410 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.599336    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv"
Jul 28 18:01:31.602572 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.602547    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 18:01:31.602671 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.602629    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-h79wl\""
Jul 28 18:01:31.602671 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.602642    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 18:01:31.619890 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.619862    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.620005 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.619928    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-stats-auth\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.620005 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.619957    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-default-certificate\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.620005 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.619981    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s54q4\" (UniqueName: \"kubernetes.io/projected/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-kube-api-access-s54q4\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.620171 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.620081    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.624339 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.624313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:31.680578 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.680550    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-7p6xv"]
Jul 28 18:01:31.690498 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.690472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:31.703239 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.703212    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-t7pms"]
Jul 28 18:01:31.703365 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.703343    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.706641 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.706616    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 18:01:31.706726 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.706616    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 18:01:31.706782 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.706670    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 18:01:31.706871 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.706855    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-g88j6\""
Jul 28 18:01:31.707062 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.707049    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 18:01:31.712699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.712680    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 18:01:31.721376 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.721357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.721457 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.721411    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.721528 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:31.721511    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:32.221495678 +0000 UTC m=+43.298057674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : configmap references non-existent config key: service-ca.crt
Jul 28 18:01:31.721596 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:31.721581    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 18:01:31.721647 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:31.721637    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:32.2216199 +0000 UTC m=+43.298181893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : secret "router-metrics-certs-default" not found
Jul 28 18:01:31.721704 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.721687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l88jc\" (UniqueName: \"kubernetes.io/projected/fa0ea14d-3bf0-4de4-ab96-2cc098eda35d-kube-api-access-l88jc\") pod \"volume-data-source-validator-549c6d54cf-l7tlv\" (UID: \"fa0ea14d-3bf0-4de4-ab96-2cc098eda35d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv"
Jul 28 18:01:31.721756 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.721734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-stats-auth\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.721808 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.721767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-default-certificate\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.721808 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.721792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s54q4\" (UniqueName: \"kubernetes.io/projected/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-kube-api-access-s54q4\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.725015 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.724998    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-default-certificate\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.725089 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.725014    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-stats-auth\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.727156 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.727140    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"]
Jul 28 18:01:31.727248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.727236    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:31.730522 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.730503    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 18:01:31.730652 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.730635    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 18:01:31.730804 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.730788    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7kwnr\""
Jul 28 18:01:31.730889 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.730796    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 18:01:31.730889 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.730831    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 18:01:31.743328 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.743311    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 18:01:31.746119 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.746103    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"]
Jul 28 18:01:31.746250 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.746233    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:31.747687 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.747663    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s54q4\" (UniqueName: \"kubernetes.io/projected/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-kube-api-access-s54q4\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:31.749853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.749835    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 18:01:31.750309 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.750292    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 18:01:31.751070 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.751052    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 18:01:31.751353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.751339    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zjhsv\""
Jul 28 18:01:31.765699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.765682    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-7p6xv"]
Jul 28 18:01:31.765776 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.765707    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"]
Jul 28 18:01:31.765776 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.765724    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"]
Jul 28 18:01:31.765776 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.765737    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-t7pms"]
Jul 28 18:01:31.765878 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.765804    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:31.768771 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.768756    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 18:01:31.770691 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.770649    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 18:01:31.770691 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.770675    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7259v\""
Jul 28 18:01:31.770916 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.770680    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 18:01:31.770916 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.770680    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 18:01:31.823077 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.823049    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lvr5\" (UniqueName: \"kubernetes.io/projected/bb04a323-a317-43c8-b604-d817792bdf73-kube-api-access-8lvr5\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.823218 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.823118    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l88jc\" (UniqueName: \"kubernetes.io/projected/fa0ea14d-3bf0-4de4-ab96-2cc098eda35d-kube-api-access-l88jc\") pod \"volume-data-source-validator-549c6d54cf-l7tlv\" (UID: \"fa0ea14d-3bf0-4de4-ab96-2cc098eda35d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv"
Jul 28 18:01:31.823218 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.823191    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb04a323-a317-43c8-b604-d817792bdf73-serving-cert\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.823325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.823236    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb04a323-a317-43c8-b604-d817792bdf73-tmp\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.823325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.823277    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bb04a323-a317-43c8-b604-d817792bdf73-snapshots\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.823325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.823313    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04a323-a317-43c8-b604-d817792bdf73-service-ca-bundle\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.823461 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.823387    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04a323-a317-43c8-b604-d817792bdf73-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.831730 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.831704    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l88jc\" (UniqueName: \"kubernetes.io/projected/fa0ea14d-3bf0-4de4-ab96-2cc098eda35d-kube-api-access-l88jc\") pod \"volume-data-source-validator-549c6d54cf-l7tlv\" (UID: \"fa0ea14d-3bf0-4de4-ab96-2cc098eda35d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv"
Jul 28 18:01:31.907723 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.907687    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv"
Jul 28 18:01:31.914916 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.914857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:31.923868 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.923840    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-serving-cert\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:31.923947 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.923880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04a323-a317-43c8-b604-d817792bdf73-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.923947 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.923929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb04a323-a317-43c8-b604-d817792bdf73-serving-cert\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.924022 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.923971    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-trusted-ca\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:31.924091 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924079    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:31.924142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924098    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blv7m\" (UniqueName: \"kubernetes.io/projected/6e2fb4b8-e83b-4597-89e1-4642b006275f-kube-api-access-blv7m\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:31.924142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924121    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ce9fc0bb-49b9-4598-9624-c944ce193418-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:31.924302 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb04a323-a317-43c8-b604-d817792bdf73-tmp\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.924302 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04a323-a317-43c8-b604-d817792bdf73-service-ca-bundle\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.924302 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924277    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmql5\" (UniqueName: \"kubernetes.io/projected/ce9fc0bb-49b9-4598-9624-c944ce193418-kube-api-access-mmql5\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:31.924436 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924303    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvr5\" (UniqueName: \"kubernetes.io/projected/bb04a323-a317-43c8-b604-d817792bdf73-kube-api-access-8lvr5\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.924436 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bb04a323-a317-43c8-b604-d817792bdf73-snapshots\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.924436 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924384    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-config\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:31.924436 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924428    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:31.924604 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924466    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn57l\" (UniqueName: \"kubernetes.io/projected/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-kube-api-access-zn57l\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:31.924604 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924515    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb04a323-a317-43c8-b604-d817792bdf73-tmp\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.924898 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924876    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04a323-a317-43c8-b604-d817792bdf73-service-ca-bundle\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.924941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924898    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04a323-a317-43c8-b604-d817792bdf73-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.924941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.924914    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bb04a323-a317-43c8-b604-d817792bdf73-snapshots\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.926221 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.926201    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb04a323-a317-43c8-b604-d817792bdf73-serving-cert\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:31.935396 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:31.935372    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lvr5\" (UniqueName: \"kubernetes.io/projected/bb04a323-a317-43c8-b604-d817792bdf73-kube-api-access-8lvr5\") pod \"insights-operator-5885fdb759-7p6xv\" (UID: \"bb04a323-a317-43c8-b604-d817792bdf73\") " pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:32.012322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.012291    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-7p6xv"
Jul 28 18:01:32.015636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.015607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:32.025580 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.025550    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmql5\" (UniqueName: \"kubernetes.io/projected/ce9fc0bb-49b9-4598-9624-c944ce193418-kube-api-access-mmql5\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:32.025705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.025614    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-config\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:32.025705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.025647    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:32.025705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.025670    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn57l\" (UniqueName: \"kubernetes.io/projected/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-kube-api-access-zn57l\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:32.025705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.025689    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-serving-cert\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:32.025930 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.025728    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-trusted-ca\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:32.025930 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.025752    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:32.025930 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.025767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:32.025930 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.025792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blv7m\" (UniqueName: \"kubernetes.io/projected/6e2fb4b8-e83b-4597-89e1-4642b006275f-kube-api-access-blv7m\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:32.025930 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.025839    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls podName:ce9fc0bb-49b9-4598-9624-c944ce193418 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:32.525799035 +0000 UTC m=+43.602361024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-zhfld" (UID: "ce9fc0bb-49b9-4598-9624-c944ce193418") : secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:32.025930 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.025875    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ce9fc0bb-49b9-4598-9624-c944ce193418-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:32.026231 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.026025    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 18:01:32.026231 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.026100    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls podName:6e2fb4b8-e83b-4597-89e1-4642b006275f nodeName:}" failed. No retries permitted until 2026-07-28 18:01:32.52608006 +0000 UTC m=+43.602642049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg8bd" (UID: "6e2fb4b8-e83b-4597-89e1-4642b006275f") : secret "samples-operator-tls" not found
Jul 28 18:01:32.026416 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.026392    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-config\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:32.026571 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.026550    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ce9fc0bb-49b9-4598-9624-c944ce193418-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:32.027150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.027126    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-trusted-ca\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:32.028273 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.028193    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-serving-cert\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:32.037162 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.037142    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmql5\" (UniqueName: \"kubernetes.io/projected/ce9fc0bb-49b9-4598-9624-c944ce193418-kube-api-access-mmql5\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:32.038249 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.038230    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn57l\" (UniqueName: \"kubernetes.io/projected/abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6-kube-api-access-zn57l\") pod \"console-operator-584dcb6b54-t7pms\" (UID: \"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6\") " pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:32.038512 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.038496    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blv7m\" (UniqueName: \"kubernetes.io/projected/6e2fb4b8-e83b-4597-89e1-4642b006275f-kube-api-access-blv7m\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:32.043733 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.043715    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:32.049505 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.049473    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv"]
Jul 28 18:01:32.052031 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:01:32.052008    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa0ea14d_3bf0_4de4_ab96_2cc098eda35d.slice/crio-e7e6fc1d4bfe6f755b214222b983e4fe63124f978a5cf5dff6734b91c7034a8c WatchSource:0}: Error finding container e7e6fc1d4bfe6f755b214222b983e4fe63124f978a5cf5dff6734b91c7034a8c: Status 404 returned error can't find the container with id e7e6fc1d4bfe6f755b214222b983e4fe63124f978a5cf5dff6734b91c7034a8c
Jul 28 18:01:32.168122 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.168090    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-7p6xv"]
Jul 28 18:01:32.170861 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:01:32.170836    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb04a323_a317_43c8_b604_d817792bdf73.slice/crio-e5f8a648ee59ac64c03b030d1e71987a3241d6dcffd3222dc1ba9a8a006559f6 WatchSource:0}: Error finding container e5f8a648ee59ac64c03b030d1e71987a3241d6dcffd3222dc1ba9a8a006559f6: Status 404 returned error can't find the container with id e5f8a648ee59ac64c03b030d1e71987a3241d6dcffd3222dc1ba9a8a006559f6
Jul 28 18:01:32.209259 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.209231    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-t7pms"]
Jul 28 18:01:32.212244 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:01:32.212209    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabbcaca7_e6ea_4ef4_b382_2d3c38ff18f6.slice/crio-55a5cf8069b5b25f635a74de295fd08d17c817a758a73aa3b9419f4a19476c97 WatchSource:0}: Error finding container 55a5cf8069b5b25f635a74de295fd08d17c817a758a73aa3b9419f4a19476c97: Status 404 returned error can't find the container with id 55a5cf8069b5b25f635a74de295fd08d17c817a758a73aa3b9419f4a19476c97
Jul 28 18:01:32.227235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.227210    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:32.227357 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.227294    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:32.227357 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.227335    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 18:01:32.227458 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.227391    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:33.22737387 +0000 UTC m=+44.303935845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : secret "router-metrics-certs-default" not found
Jul 28 18:01:32.227458 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.227428    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:33.227411488 +0000 UTC m=+44.303973479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : configmap references non-existent config key: service-ca.crt
Jul 28 18:01:32.529929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.529883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:32.530378 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.529998    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:32.530378 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.530061    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:32.530378 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.530082    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 18:01:32.530378 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.530132    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls podName:6e2fb4b8-e83b-4597-89e1-4642b006275f nodeName:}" failed. No retries permitted until 2026-07-28 18:01:33.530118012 +0000 UTC m=+44.606679987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg8bd" (UID: "6e2fb4b8-e83b-4597-89e1-4642b006275f") : secret "samples-operator-tls" not found
Jul 28 18:01:32.530378 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.530145    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls podName:ce9fc0bb-49b9-4598-9624-c944ce193418 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:33.530139532 +0000 UTC m=+44.606701507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-zhfld" (UID: "ce9fc0bb-49b9-4598-9624-c944ce193418") : secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:32.594360 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.594327    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" event={"ID":"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6","Type":"ContainerStarted","Data":"55a5cf8069b5b25f635a74de295fd08d17c817a758a73aa3b9419f4a19476c97"}
Jul 28 18:01:32.595423 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.595389    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv" event={"ID":"fa0ea14d-3bf0-4de4-ab96-2cc098eda35d","Type":"ContainerStarted","Data":"e7e6fc1d4bfe6f755b214222b983e4fe63124f978a5cf5dff6734b91c7034a8c"}
Jul 28 18:01:32.596312 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.596284    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-7p6xv" event={"ID":"bb04a323-a317-43c8-b604-d817792bdf73","Type":"ContainerStarted","Data":"e5f8a648ee59ac64c03b030d1e71987a3241d6dcffd3222dc1ba9a8a006559f6"}
Jul 28 18:01:32.630892 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.630863    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:32.631046 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:32.630912    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:32.631102 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.631039    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 18:01:32.631102 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.631063    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b567bb8ff-8p6pr: secret "image-registry-tls" not found
Jul 28 18:01:32.631184 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.631128    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls podName:2e61e8b3-e5cd-4e79-ab56-47a81061fdc8 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:40.631108611 +0000 UTC m=+51.707670604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls") pod "image-registry-5b567bb8ff-8p6pr" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8") : secret "image-registry-tls" not found
Jul 28 18:01:32.631184 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.631128    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 18:01:32.631272 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:32.631186    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls podName:fd9c632b-0b01-4a1d-a636-5ce8ec009f9c nodeName:}" failed. No retries permitted until 2026-07-28 18:01:40.63116858 +0000 UTC m=+51.707730558 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls") pod "dns-default-ftg8z" (UID: "fd9c632b-0b01-4a1d-a636-5ce8ec009f9c") : secret "dns-default-metrics-tls" not found
Jul 28 18:01:33.237354 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:33.237315    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:33.237525 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:33.237461    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 18:01:33.237585 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:33.237538    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:35.23751666 +0000 UTC m=+46.314078651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : secret "router-metrics-certs-default" not found
Jul 28 18:01:33.237989 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:33.237960    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:33.238201 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:33.238176    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:35.238162172 +0000 UTC m=+46.314724154 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : configmap references non-existent config key: service-ca.crt
Jul 28 18:01:33.540968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:33.540889    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:33.540968 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:33.540955    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 18:01:33.541459 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:33.541042    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls podName:6e2fb4b8-e83b-4597-89e1-4642b006275f nodeName:}" failed. No retries permitted until 2026-07-28 18:01:35.54102117 +0000 UTC m=+46.617583159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg8bd" (UID: "6e2fb4b8-e83b-4597-89e1-4642b006275f") : secret "samples-operator-tls" not found
Jul 28 18:01:33.541459 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:33.541079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:33.541459 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:33.541223    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:33.541459 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:33.541288    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls podName:ce9fc0bb-49b9-4598-9624-c944ce193418 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:35.541271994 +0000 UTC m=+46.617833971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-zhfld" (UID: "ce9fc0bb-49b9-4598-9624-c944ce193418") : secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:34.602522 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:34.602481    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv" event={"ID":"fa0ea14d-3bf0-4de4-ab96-2cc098eda35d","Type":"ContainerStarted","Data":"e40d92647edbaee0d60235bdd60201828d7a0e8264ae68c6a2d3ecdc46b625db"}
Jul 28 18:01:34.625664 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:34.625603    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-l7tlv" podStartSLOduration=2.034268161 podStartE2EDuration="3.625582015s" totalImagesPullingTime="1.591313854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:34.624347224 +0000 UTC m=+45.700909220" watchObservedRunningTime="2026-07-28 18:01:34.625582015 +0000 UTC m=+45.702144009"
Jul 28 18:01:34.705235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:34.705202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:34.759122 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:34.759086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:34.914230 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:34.914146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:35.258714 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:35.258690    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:35.258860 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:35.258840    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 18:01:35.258933 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:35.258845    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:35.258933 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:35.258910    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:39.258895035 +0000 UTC m=+50.335457013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : secret "router-metrics-certs-default" not found
Jul 28 18:01:35.259019 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:35.258959    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:39.258935089 +0000 UTC m=+50.335497076 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : configmap references non-existent config key: service-ca.crt
Jul 28 18:01:35.562478 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:35.562391    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:35.562625 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:35.562541    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 18:01:35.562625 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:35.562605    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls podName:6e2fb4b8-e83b-4597-89e1-4642b006275f nodeName:}" failed. No retries permitted until 2026-07-28 18:01:39.56259143 +0000 UTC m=+50.639153408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg8bd" (UID: "6e2fb4b8-e83b-4597-89e1-4642b006275f") : secret "samples-operator-tls" not found
Jul 28 18:01:35.562625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:35.562600    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:35.562794 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:35.562780    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:35.562873 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:35.562863    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls podName:ce9fc0bb-49b9-4598-9624-c944ce193418 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:39.562847334 +0000 UTC m=+50.639409312 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-zhfld" (UID: "ce9fc0bb-49b9-4598-9624-c944ce193418") : secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:35.606369 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:35.606334    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-7p6xv" event={"ID":"bb04a323-a317-43c8-b604-d817792bdf73","Type":"ContainerStarted","Data":"dce4311b7f8fcb43ea6a1a9f5d092d950d9cd66fc116c22fded4e10c22298612"}
Jul 28 18:01:35.607990 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:35.607972    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/0.log"
Jul 28 18:01:35.608091 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:35.608005    2583 generic.go:356] "Generic (PLEG): container finished" podID="abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6" containerID="87a43b57700da129e7af42f82ca3e630d0b9a325cdb6fbeab3bff417e865e99d" exitCode=255
Jul 28 18:01:35.608149 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:35.608097    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" event={"ID":"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6","Type":"ContainerDied","Data":"87a43b57700da129e7af42f82ca3e630d0b9a325cdb6fbeab3bff417e865e99d"}
Jul 28 18:01:35.608291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:35.608274    2583 scope.go:117] "RemoveContainer" containerID="87a43b57700da129e7af42f82ca3e630d0b9a325cdb6fbeab3bff417e865e99d"
Jul 28 18:01:35.624611 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:35.624568    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-7p6xv" podStartSLOduration=1.584824921 podStartE2EDuration="4.624554794s" totalImagesPullingTime="3.039729873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:35.624176448 +0000 UTC m=+46.700738447" watchObservedRunningTime="2026-07-28 18:01:35.624554794 +0000 UTC m=+46.701116798"
Jul 28 18:01:36.405756 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.405719    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:36.506850 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.506807    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:36.613629 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.613600    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/1.log"
Jul 28 18:01:36.614156 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.614139    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/0.log"
Jul 28 18:01:36.614223 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.614180    2583 generic.go:356] "Generic (PLEG): container finished" podID="abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6" containerID="ff9cfd789f7a3e88cf74cd71e33fb7782671a8338ec25d5bb6c92024efdb4145" exitCode=255
Jul 28 18:01:36.614297 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.614278    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" event={"ID":"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6","Type":"ContainerDied","Data":"ff9cfd789f7a3e88cf74cd71e33fb7782671a8338ec25d5bb6c92024efdb4145"}
Jul 28 18:01:36.614375 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.614332    2583 scope.go:117] "RemoveContainer" containerID="87a43b57700da129e7af42f82ca3e630d0b9a325cdb6fbeab3bff417e865e99d"
Jul 28 18:01:36.614795 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.614768    2583 scope.go:117] "RemoveContainer" containerID="ff9cfd789f7a3e88cf74cd71e33fb7782671a8338ec25d5bb6c92024efdb4145"
Jul 28 18:01:36.615037 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:36.615011    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-t7pms_openshift-console-operator(abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" podUID="abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6"
Jul 28 18:01:36.626161 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.626130    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5nxds"]
Jul 28 18:01:36.655374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.655330    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5nxds"]
Jul 28 18:01:36.655514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.655444    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.657936 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.657911    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:36.658678 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.658658    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4jjmm\""
Jul 28 18:01:36.658678 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.658676    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 18:01:36.658918 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.658902    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 18:01:36.672182 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.672156    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.672421 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.672399    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10f0792e-73a1-45ed-93fd-508e7885060d-data-volume\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.672564 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.672525    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vdj8\" (UniqueName: \"kubernetes.io/projected/10f0792e-73a1-45ed-93fd-508e7885060d-kube-api-access-8vdj8\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.672641 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.672624    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.672706 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.672696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10f0792e-73a1-45ed-93fd-508e7885060d-crio-socket\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.722362 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.722322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:36.773572 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.773534    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10f0792e-73a1-45ed-93fd-508e7885060d-data-volume\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.773758 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.773613    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vdj8\" (UniqueName: \"kubernetes.io/projected/10f0792e-73a1-45ed-93fd-508e7885060d-kube-api-access-8vdj8\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.773758 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.773641    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.773758 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.773665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10f0792e-73a1-45ed-93fd-508e7885060d-crio-socket\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.773950 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:36.773768    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 18:01:36.773950 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.773858    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10f0792e-73a1-45ed-93fd-508e7885060d-crio-socket\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.773950 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:36.773870    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm podName:10f0792e-73a1-45ed-93fd-508e7885060d nodeName:}" failed. No retries permitted until 2026-07-28 18:01:37.27384484 +0000 UTC m=+48.350406829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5nxds" (UID: "10f0792e-73a1-45ed-93fd-508e7885060d") : configmap "kube-rbac-proxy" not found
Jul 28 18:01:36.773950 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.773915    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.773950 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.773945    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10f0792e-73a1-45ed-93fd-508e7885060d-data-volume\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:36.774188 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:36.774083    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:36.774188 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:36.774138    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls podName:10f0792e-73a1-45ed-93fd-508e7885060d nodeName:}" failed. No retries permitted until 2026-07-28 18:01:37.274124257 +0000 UTC m=+48.350686236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5nxds" (UID: "10f0792e-73a1-45ed-93fd-508e7885060d") : secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:36.782926 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:36.782903    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vdj8\" (UniqueName: \"kubernetes.io/projected/10f0792e-73a1-45ed-93fd-508e7885060d-kube-api-access-8vdj8\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:37.277217 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:37.277177    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:37.277386 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:37.277275    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:37.277386 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:37.277296    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 18:01:37.277386 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:37.277360    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm podName:10f0792e-73a1-45ed-93fd-508e7885060d nodeName:}" failed. No retries permitted until 2026-07-28 18:01:38.277344812 +0000 UTC m=+49.353906786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5nxds" (UID: "10f0792e-73a1-45ed-93fd-508e7885060d") : configmap "kube-rbac-proxy" not found
Jul 28 18:01:37.277493 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:37.277412    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:37.277493 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:37.277487    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls podName:10f0792e-73a1-45ed-93fd-508e7885060d nodeName:}" failed. No retries permitted until 2026-07-28 18:01:38.277472143 +0000 UTC m=+49.354034131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5nxds" (UID: "10f0792e-73a1-45ed-93fd-508e7885060d") : secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:37.618667 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:37.618585    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/1.log"
Jul 28 18:01:37.619031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:37.618966    2583 scope.go:117] "RemoveContainer" containerID="ff9cfd789f7a3e88cf74cd71e33fb7782671a8338ec25d5bb6c92024efdb4145"
Jul 28 18:01:37.619180 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:37.619159    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-t7pms_openshift-console-operator(abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" podUID="abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6"
Jul 28 18:01:38.219017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:38.218992    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dfk6x_d8a76901-42f5-413e-8fa4-64c94fe7bcf7/dns-node-resolver/0.log"
Jul 28 18:01:38.285766 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:38.285735    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:38.285921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:38.285833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:38.285921 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:38.285882    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 18:01:38.286041 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:38.285954    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm podName:10f0792e-73a1-45ed-93fd-508e7885060d nodeName:}" failed. No retries permitted until 2026-07-28 18:01:40.285935291 +0000 UTC m=+51.362497287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5nxds" (UID: "10f0792e-73a1-45ed-93fd-508e7885060d") : configmap "kube-rbac-proxy" not found
Jul 28 18:01:38.286041 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:38.285971    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:38.286041 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:38.286038    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls podName:10f0792e-73a1-45ed-93fd-508e7885060d nodeName:}" failed. No retries permitted until 2026-07-28 18:01:40.286021776 +0000 UTC m=+51.362583751 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5nxds" (UID: "10f0792e-73a1-45ed-93fd-508e7885060d") : secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:39.218023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:39.217994    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qz6v8_6887fd7f-e61b-49b4-8581-a50259ab2be2/node-ca/0.log"
Jul 28 18:01:39.295011 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:39.294972    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:39.295120 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:39.295062    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:39.295169 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:39.295121    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 18:01:39.295207 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:39.295188    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:47.295172751 +0000 UTC m=+58.371734726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : secret "router-metrics-certs-default" not found
Jul 28 18:01:39.295255 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:39.295215    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:47.295200755 +0000 UTC m=+58.371762735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : configmap references non-existent config key: service-ca.crt
Jul 28 18:01:39.598606 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:39.598516    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:39.598751 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:39.598609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:39.598751 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:39.598668    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 18:01:39.598751 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:39.598695    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:39.598751 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:39.598748    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls podName:6e2fb4b8-e83b-4597-89e1-4642b006275f nodeName:}" failed. No retries permitted until 2026-07-28 18:01:47.5987323 +0000 UTC m=+58.675294275 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg8bd" (UID: "6e2fb4b8-e83b-4597-89e1-4642b006275f") : secret "samples-operator-tls" not found
Jul 28 18:01:39.598905 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:39.598765    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls podName:ce9fc0bb-49b9-4598-9624-c944ce193418 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:47.598757631 +0000 UTC m=+58.675319606 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-zhfld" (UID: "ce9fc0bb-49b9-4598-9624-c944ce193418") : secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:40.016945 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:40.016909    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wtkbg_b3fb8585-f057-4f84-9576-14df70802b01/serve-healthcheck-canary/0.log"
Jul 28 18:01:40.204236 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:40.204193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:40.304373 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:40.304288    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:40.304716 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:40.304488    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:40.304716 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:40.304614    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:40.304716 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:40.304668    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls podName:10f0792e-73a1-45ed-93fd-508e7885060d nodeName:}" failed. No retries permitted until 2026-07-28 18:01:44.304655172 +0000 UTC m=+55.381217147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5nxds" (UID: "10f0792e-73a1-45ed-93fd-508e7885060d") : secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:40.304883 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:40.304866    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10f0792e-73a1-45ed-93fd-508e7885060d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:40.677474 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:40.677391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:40.708626 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:40.708590    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:40.708740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:40.708639    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:40.708777 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:40.708747    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 18:01:40.708777 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:40.708766    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b567bb8ff-8p6pr: secret "image-registry-tls" not found
Jul 28 18:01:40.708868 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:40.708837    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls podName:2e61e8b3-e5cd-4e79-ab56-47a81061fdc8 nodeName:}" failed. No retries permitted until 2026-07-28 18:01:56.708801334 +0000 UTC m=+67.785363313 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls") pod "image-registry-5b567bb8ff-8p6pr" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8") : secret "image-registry-tls" not found
Jul 28 18:01:40.708868 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:40.708745    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 18:01:40.708935 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:40.708878    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls podName:fd9c632b-0b01-4a1d-a636-5ce8ec009f9c nodeName:}" failed. No retries permitted until 2026-07-28 18:01:56.708867359 +0000 UTC m=+67.785429333 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls") pod "dns-default-ftg8z" (UID: "fd9c632b-0b01-4a1d-a636-5ce8ec009f9c") : secret "dns-default-metrics-tls" not found
Jul 28 18:01:40.777867 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:40.777833    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:42.044888 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:42.044845    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:42.045251 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:42.044931    2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:01:42.045251 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:42.045244    2583 scope.go:117] "RemoveContainer" containerID="ff9cfd789f7a3e88cf74cd71e33fb7782671a8338ec25d5bb6c92024efdb4145"
Jul 28 18:01:42.045401 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:42.045380    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-t7pms_openshift-console-operator(abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" podUID="abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6"
Jul 28 18:01:42.309344 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:42.309261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:42.409509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:42.409478    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:42.504877 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:42.504846    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:42.609580 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:42.609498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:42.673406 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:42.673375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:43.270044 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:43.270007    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:43.362231 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:43.362196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:44.339874 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:44.339837    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:44.340359 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:44.339974    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:44.340359 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:44.340049    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls podName:10f0792e-73a1-45ed-93fd-508e7885060d nodeName:}" failed. No retries permitted until 2026-07-28 18:01:52.34002772 +0000 UTC m=+63.416589709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5nxds" (UID: "10f0792e-73a1-45ed-93fd-508e7885060d") : secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:44.556865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:44.556828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:44.621053 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:44.620968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:44.901846 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:44.901749    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:45.002881 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:45.002847    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:45.103978 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:45.103948    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:46.096103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:46.096073    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:47.319408 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:47.319374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:47.366001 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:47.365965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:47.366171 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:47.366029    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:01:47.366171 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:47.366122    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 18:01:47.366254 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:47.366184    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:02:03.366168132 +0000 UTC m=+74.442730110 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : secret "router-metrics-certs-default" not found
Jul 28 18:01:47.366254 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:47.366197    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle podName:e6b6c4c5-a304-4406-87eb-e99a3e1f3453 nodeName:}" failed. No retries permitted until 2026-07-28 18:02:03.366191752 +0000 UTC m=+74.442753727 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle") pod "router-default-7cb56ffbff-lv2ps" (UID: "e6b6c4c5-a304-4406-87eb-e99a3e1f3453") : configmap references non-existent config key: service-ca.crt
Jul 28 18:01:47.417031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:47.417000    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:47.655138 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:47.655043    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:47.668271 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:47.668230    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:01:47.668420 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:47.668312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:01:47.668420 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:47.668383    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:47.668420 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:47.668396    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 18:01:47.668563 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:47.668446    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls podName:6e2fb4b8-e83b-4597-89e1-4642b006275f nodeName:}" failed. No retries permitted until 2026-07-28 18:02:03.668432611 +0000 UTC m=+74.744994586 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg8bd" (UID: "6e2fb4b8-e83b-4597-89e1-4642b006275f") : secret "samples-operator-tls" not found
Jul 28 18:01:47.668563 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:47.668458    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls podName:ce9fc0bb-49b9-4598-9624-c944ce193418 nodeName:}" failed. No retries permitted until 2026-07-28 18:02:03.668452707 +0000 UTC m=+74.745014682 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-zhfld" (UID: "ce9fc0bb-49b9-4598-9624-c944ce193418") : secret "cluster-monitoring-operator-tls" not found
Jul 28 18:01:47.756826 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:47.756780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:47.949684 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:47.949642    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:48.050549 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:48.050511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:48.116475 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:48.116419    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:48.627382 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:48.627352    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:48.851150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:48.851116    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:49.050612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:49.050581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:49.833395 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:49.833358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:50.027467 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:50.027432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:50.440751 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:50.440716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:50.625635 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:50.625599    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:51.561285 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:51.561254    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b2tvv"
Jul 28 18:01:52.102280 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:52.102239    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:52.411061 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:52.410981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:01:52.411205 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:52.411128    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:52.411205 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:52.411188    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls podName:10f0792e-73a1-45ed-93fd-508e7885060d nodeName:}" failed. No retries permitted until 2026-07-28 18:02:08.411173604 +0000 UTC m=+79.487735579 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5nxds" (UID: "10f0792e-73a1-45ed-93fd-508e7885060d") : secret "insights-runtime-extractor-tls" not found
Jul 28 18:01:53.365104 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:53.365066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:53.464989 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:53.464949    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:53.564981 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:53.564948    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:53.666613 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:53.666528    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:54.028355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:54.028325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:54.225002 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:54.224967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:54.619871 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:54.619832    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:55.130774 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.130733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:55.133827 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.133799    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 18:01:55.143272 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.143243    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c98a16c-de7a-4336-b23c-fcb14ec96265-metrics-certs\") pod \"network-metrics-daemon-4v2hh\" (UID: \"3c98a16c-de7a-4336-b23c-fcb14ec96265\") " pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:55.218350 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.218319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:55.222044 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.222020    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x879t\""
Jul 28 18:01:55.229742 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.229722    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v2hh"
Jul 28 18:01:55.231090 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.231050    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4fw\" (UniqueName: \"kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw\") pod \"network-check-target-kbqwj\" (UID: \"f10871a0-d4c5-4c63-8dc7-8f5c5123a523\") " pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:55.231249 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.231231    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:55.234423 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.234403    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 18:01:55.234519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.234411    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 18:01:55.244096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.244068    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d6960ee-cc72-488e-bae6-23608c90ab15-original-pull-secret\") pod \"global-pull-secret-syncer-tc422\" (UID: \"1d6960ee-cc72-488e-bae6-23608c90ab15\") " pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:55.245975 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.245951    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 18:01:55.254963 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.254935    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx4fw\" (UniqueName: \"kubernetes.io/projected/f10871a0-d4c5-4c63-8dc7-8f5c5123a523-kube-api-access-vx4fw\") pod \"network-check-target-kbqwj\" (UID: \"f10871a0-d4c5-4c63-8dc7-8f5c5123a523\") " pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:55.355212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.355180    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4v2hh"]
Jul 28 18:01:55.358059 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:01:55.358029    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c98a16c_de7a_4336_b23c_fcb14ec96265.slice/crio-849496a06b6941e5af8c5010b5407e1be25916bbd32dfd934abff943833228b0 WatchSource:0}: Error finding container 849496a06b6941e5af8c5010b5407e1be25916bbd32dfd934abff943833228b0: Status 404 returned error can't find the container with id 849496a06b6941e5af8c5010b5407e1be25916bbd32dfd934abff943833228b0
Jul 28 18:01:55.400132 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.400055    2583 scope.go:117] "RemoveContainer" containerID="ff9cfd789f7a3e88cf74cd71e33fb7782671a8338ec25d5bb6c92024efdb4145"
Jul 28 18:01:55.512528 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.512491    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc422"
Jul 28 18:01:55.527454 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.527430    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9p4qw\""
Jul 28 18:01:55.534838 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.534506    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:01:55.640458 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.640426    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tc422"]
Jul 28 18:01:55.643087 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:01:55.643051    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6960ee_cc72_488e_bae6_23608c90ab15.slice/crio-fca964d85052c15cdcfcf8e4399602b3d5e9365f93131cc11745835d7850d34c WatchSource:0}: Error finding container fca964d85052c15cdcfcf8e4399602b3d5e9365f93131cc11745835d7850d34c: Status 404 returned error can't find the container with id fca964d85052c15cdcfcf8e4399602b3d5e9365f93131cc11745835d7850d34c
Jul 28 18:01:55.665588 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.665508    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4v2hh" event={"ID":"3c98a16c-de7a-4336-b23c-fcb14ec96265","Type":"ContainerStarted","Data":"849496a06b6941e5af8c5010b5407e1be25916bbd32dfd934abff943833228b0"}
Jul 28 18:01:55.667151 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.667128    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:01:55.667568 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.667552    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/1.log"
Jul 28 18:01:55.667684 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.667584    2583 generic.go:356] "Generic (PLEG): container finished" podID="abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6" containerID="49a45c39317c21dbcf2f2c58a0320c064deb8bc413f05c13706e75726be2036f" exitCode=255
Jul 28 18:01:55.667684 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.667646    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" event={"ID":"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6","Type":"ContainerDied","Data":"49a45c39317c21dbcf2f2c58a0320c064deb8bc413f05c13706e75726be2036f"}
Jul 28 18:01:55.667793 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.667695    2583 scope.go:117] "RemoveContainer" containerID="ff9cfd789f7a3e88cf74cd71e33fb7782671a8338ec25d5bb6c92024efdb4145"
Jul 28 18:01:55.668128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.668062    2583 scope.go:117] "RemoveContainer" containerID="49a45c39317c21dbcf2f2c58a0320c064deb8bc413f05c13706e75726be2036f"
Jul 28 18:01:55.668247 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.668219    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kbqwj"]
Jul 28 18:01:55.668359 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:01:55.668280    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-t7pms_openshift-console-operator(abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" podUID="abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6"
Jul 28 18:01:55.669066 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.668983    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tc422" event={"ID":"1d6960ee-cc72-488e-bae6-23608c90ab15","Type":"ContainerStarted","Data":"fca964d85052c15cdcfcf8e4399602b3d5e9365f93131cc11745835d7850d34c"}
Jul 28 18:01:55.671011 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:01:55.670991    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf10871a0_d4c5_4c63_8dc7_8f5c5123a523.slice/crio-70f183aeed3480e6be57b1ce12b6bee1162864110a081311ca3d333b57d93ebf WatchSource:0}: Error finding container 70f183aeed3480e6be57b1ce12b6bee1162864110a081311ca3d333b57d93ebf: Status 404 returned error can't find the container with id 70f183aeed3480e6be57b1ce12b6bee1162864110a081311ca3d333b57d93ebf
Jul 28 18:01:55.826180 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:55.826129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:56.023982 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.023946    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:56.427402 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.427369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:56.644127 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.643996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:56.675336 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.675291    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4v2hh" event={"ID":"3c98a16c-de7a-4336-b23c-fcb14ec96265","Type":"ContainerStarted","Data":"36a9a496f32f417b5a9236d7513d56864f28e5f94ff4152671139379d0f0b22b"}
Jul 28 18:01:56.675519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.675386    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4v2hh" event={"ID":"3c98a16c-de7a-4336-b23c-fcb14ec96265","Type":"ContainerStarted","Data":"7f156f0beee4804fdd9986e32a5f051dc12dd4ee9d68ac38186fc1f87302bc88"}
Jul 28 18:01:56.677063 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.677032    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kbqwj" event={"ID":"f10871a0-d4c5-4c63-8dc7-8f5c5123a523","Type":"ContainerStarted","Data":"70f183aeed3480e6be57b1ce12b6bee1162864110a081311ca3d333b57d93ebf"}
Jul 28 18:01:56.678879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.678857    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:01:56.698515 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.698454    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4v2hh" podStartSLOduration=66.834418353 podStartE2EDuration="1m7.698437944s" totalImagesPullingTime="864.019591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:56.696631319 +0000 UTC m=+67.773193328" watchObservedRunningTime="2026-07-28 18:01:56.698437944 +0000 UTC m=+67.774999941"
Jul 28 18:01:56.744979 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.744936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:56.745167 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.745001    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:56.747965 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.747941    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") pod \"image-registry-5b567bb8ff-8p6pr\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") " pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:56.748084 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.747963    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd9c632b-0b01-4a1d-a636-5ce8ec009f9c-metrics-tls\") pod \"dns-default-ftg8z\" (UID: \"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c\") " pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:56.899022 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.898944    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8fv8h\""
Jul 28 18:01:56.906753 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.906710    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:56.911179 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.911155    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g9rrw\""
Jul 28 18:01:56.918832 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:56.918785    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:01:57.086080 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:57.086044    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b567bb8ff-8p6pr"]
Jul 28 18:01:57.089656 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:01:57.089623    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e61e8b3_e5cd_4e79_ab56_47a81061fdc8.slice/crio-d9586c69792d0b1435aedc71ad75c17d367203c26ca3dd1034e7bf29b9b389a8 WatchSource:0}: Error finding container d9586c69792d0b1435aedc71ad75c17d367203c26ca3dd1034e7bf29b9b389a8: Status 404 returned error can't find the container with id d9586c69792d0b1435aedc71ad75c17d367203c26ca3dd1034e7bf29b9b389a8
Jul 28 18:01:57.105223 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:57.105186    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ftg8z"]
Jul 28 18:01:57.111130 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:01:57.111095    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd9c632b_0b01_4a1d_a636_5ce8ec009f9c.slice/crio-3577f942f92aac883fa8774955ebebe2fbcb9f812294036e45d49de9b0b0a7ee WatchSource:0}: Error finding container 3577f942f92aac883fa8774955ebebe2fbcb9f812294036e45d49de9b0b0a7ee: Status 404 returned error can't find the container with id 3577f942f92aac883fa8774955ebebe2fbcb9f812294036e45d49de9b0b0a7ee
Jul 28 18:01:57.227757 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:57.227720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:57.624541 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:57.624461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:57.685862 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:57.685749    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ftg8z" event={"ID":"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c","Type":"ContainerStarted","Data":"3577f942f92aac883fa8774955ebebe2fbcb9f812294036e45d49de9b0b0a7ee"}
Jul 28 18:01:57.688848 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:57.688661    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr" event={"ID":"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8","Type":"ContainerStarted","Data":"c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb"}
Jul 28 18:01:57.688848 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:57.688697    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr" event={"ID":"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8","Type":"ContainerStarted","Data":"d9586c69792d0b1435aedc71ad75c17d367203c26ca3dd1034e7bf29b9b389a8"}
Jul 28 18:01:57.688848 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:57.688727    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:01:57.711719 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:57.711660    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr" podStartSLOduration=67.711637005 podStartE2EDuration="1m7.711637005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:57.710838185 +0000 UTC m=+68.787400183" watchObservedRunningTime="2026-07-28 18:01:57.711637005 +0000 UTC m=+68.788198985"
Jul 28 18:01:57.825747 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:57.825678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:58.021165 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:58.021129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:01:59.959215 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:01:59.959175    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:00.025950 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:00.025913    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:01.702098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:01.702058    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kbqwj" event={"ID":"f10871a0-d4c5-4c63-8dc7-8f5c5123a523","Type":"ContainerStarted","Data":"912aeb1ff66db65007ddf04152036eda6204edb586b608f9d77cff6f6e0970a7"}
Jul 28 18:02:01.702586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:01.702238    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:02:01.703463 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:01.703439    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tc422" event={"ID":"1d6960ee-cc72-488e-bae6-23608c90ab15","Type":"ContainerStarted","Data":"6d65c156c7ea6afe9e268219b63de874f4a56de45401ab28b239d702f439bb58"}
Jul 28 18:02:01.706523 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:01.706483    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ftg8z" event={"ID":"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c","Type":"ContainerStarted","Data":"931b55121816fc355c6e485d38d57f218f0d78ca9c27d96fde2e2ce962bdd8cb"}
Jul 28 18:02:01.706663 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:01.706565    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ftg8z" event={"ID":"fd9c632b-0b01-4a1d-a636-5ce8ec009f9c","Type":"ContainerStarted","Data":"8e2f753e227c4ada43a11b580b60294d8d74b1df22bd41d2a1bce1cc9bf19d6f"}
Jul 28 18:02:01.706860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:01.706839    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:02:01.721333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:01.721288    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kbqwj" podStartSLOduration=67.591506027 podStartE2EDuration="1m12.721272882s" totalImagesPullingTime="5.129766855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:01.720142982 +0000 UTC m=+72.796704980" watchObservedRunningTime="2026-07-28 18:02:01.721272882 +0000 UTC m=+72.797834879"
Jul 28 18:02:01.737666 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:01.737616    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tc422" podStartSLOduration=66.566768215 podStartE2EDuration="1m11.737601668s" totalImagesPullingTime="5.170833453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:01.736538266 +0000 UTC m=+72.813100265" watchObservedRunningTime="2026-07-28 18:02:01.737601668 +0000 UTC m=+72.814163697"
Jul 28 18:02:01.756774 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:01.756732    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ftg8z" podStartSLOduration=34.065569805 podStartE2EDuration="37.756722055s" totalImagesPullingTime="3.69115225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:01:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:01.755969539 +0000 UTC m=+72.832531537" watchObservedRunningTime="2026-07-28 18:02:01.756722055 +0000 UTC m=+72.833284052"
Jul 28 18:02:02.043926 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:02.043841    2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:02:02.043926 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:02.043900    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:02:02.044227 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:02.044214    2583 scope.go:117] "RemoveContainer" containerID="49a45c39317c21dbcf2f2c58a0320c064deb8bc413f05c13706e75726be2036f"
Jul 28 18:02:02.044387 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:02:02.044369    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-t7pms_openshift-console-operator(abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" podUID="abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6"
Jul 28 18:02:03.406422 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.406390    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:02:03.406783 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.406446    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:02:03.407035 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.407018    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-service-ca-bundle\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:02:03.408659 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.408640    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b6c4c5-a304-4406-87eb-e99a3e1f3453-metrics-certs\") pod \"router-default-7cb56ffbff-lv2ps\" (UID: \"e6b6c4c5-a304-4406-87eb-e99a3e1f3453\") " pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:02:03.623519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.623489    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-h6vph\""
Jul 28 18:02:03.631292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.631273    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:02:03.708340 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.708266    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:02:03.708512 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.708380    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:02:03.711404 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.711360    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e2fb4b8-e83b-4597-89e1-4642b006275f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg8bd\" (UID: \"6e2fb4b8-e83b-4597-89e1-4642b006275f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:02:03.711404 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.711357    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ce9fc0bb-49b9-4598-9624-c944ce193418-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-zhfld\" (UID: \"ce9fc0bb-49b9-4598-9624-c944ce193418\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:02:03.791201 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.791167    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cb56ffbff-lv2ps"]
Jul 28 18:02:03.794117 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:02:03.794086    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b6c4c5_a304_4406_87eb_e99a3e1f3453.slice/crio-771d7db0abdb97d50e01ccb605deecb34120efd880c1196ff2702067f794bf46 WatchSource:0}: Error finding container 771d7db0abdb97d50e01ccb605deecb34120efd880c1196ff2702067f794bf46: Status 404 returned error can't find the container with id 771d7db0abdb97d50e01ccb605deecb34120efd880c1196ff2702067f794bf46
Jul 28 18:02:03.862230 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.862201    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zjhsv\""
Jul 28 18:02:03.869827 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.869792    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"
Jul 28 18:02:03.875981 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.875956    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7259v\""
Jul 28 18:02:03.884187 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:03.884165    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"
Jul 28 18:02:04.005134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:04.005096    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd"]
Jul 28 18:02:04.020934 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:04.020902    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld"]
Jul 28 18:02:04.023777 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:02:04.023751    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce9fc0bb_49b9_4598_9624_c944ce193418.slice/crio-244ea355fac1d2460f44d59e4c8731f6f1c13f27ceda9b6efe182e3ac6baf6d8 WatchSource:0}: Error finding container 244ea355fac1d2460f44d59e4c8731f6f1c13f27ceda9b6efe182e3ac6baf6d8: Status 404 returned error can't find the container with id 244ea355fac1d2460f44d59e4c8731f6f1c13f27ceda9b6efe182e3ac6baf6d8
Jul 28 18:02:04.718975 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:04.718939    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld" event={"ID":"ce9fc0bb-49b9-4598-9624-c944ce193418","Type":"ContainerStarted","Data":"244ea355fac1d2460f44d59e4c8731f6f1c13f27ceda9b6efe182e3ac6baf6d8"}
Jul 28 18:02:04.720224 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:04.720196    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cb56ffbff-lv2ps" event={"ID":"e6b6c4c5-a304-4406-87eb-e99a3e1f3453","Type":"ContainerStarted","Data":"aec4f87bb857fdbcc07fd91be8836e198f8496a07b46e58b4771b69cfc1220f1"}
Jul 28 18:02:04.720338 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:04.720245    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cb56ffbff-lv2ps" event={"ID":"e6b6c4c5-a304-4406-87eb-e99a3e1f3453","Type":"ContainerStarted","Data":"771d7db0abdb97d50e01ccb605deecb34120efd880c1196ff2702067f794bf46"}
Jul 28 18:02:04.721196 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:04.721178    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd" event={"ID":"6e2fb4b8-e83b-4597-89e1-4642b006275f","Type":"ContainerStarted","Data":"d3d89a79db4e489a4a3ea930214585708bd1c74a2528df2bd70a55a92988c123"}
Jul 28 18:02:04.750521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:04.750469    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7cb56ffbff-lv2ps" podStartSLOduration=33.750449371 podStartE2EDuration="33.750449371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:01:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:04.750302592 +0000 UTC m=+75.826864590" watchObservedRunningTime="2026-07-28 18:02:04.750449371 +0000 UTC m=+75.827011371"
Jul 28 18:02:05.632322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:05.632275    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:02:05.635170 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:05.635145    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:02:05.725404 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:05.725373    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:02:05.726784 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:05.726761    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7cb56ffbff-lv2ps"
Jul 28 18:02:06.447081 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:06.447040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:06.549237 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:06.549176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:06.647733 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:06.647676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:06.730555 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:06.730460    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd" event={"ID":"6e2fb4b8-e83b-4597-89e1-4642b006275f","Type":"ContainerStarted","Data":"43a8864df529b43d10d3109cc9c8a8078b6cb9ac641440ee6eae23e47a6804cc"}
Jul 28 18:02:06.731023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:06.730553    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd" event={"ID":"6e2fb4b8-e83b-4597-89e1-4642b006275f","Type":"ContainerStarted","Data":"316fd2ca38fa7e19dc81a9f2166332f42540f737db5ba1f1e33d32ee5beeb26c"}
Jul 28 18:02:06.731959 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:06.731937    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld" event={"ID":"ce9fc0bb-49b9-4598-9624-c944ce193418","Type":"ContainerStarted","Data":"85ef4268bb418d15b88fa4386c0607f5fe7a8fbb2aa1df7f9470d71bb266bf7a"}
Jul 28 18:02:06.753416 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:06.753354    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg8bd" podStartSLOduration=33.680374695 podStartE2EDuration="35.753335456s" totalImagesPullingTime="2.072960761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:06.752245431 +0000 UTC m=+77.828807427" watchObservedRunningTime="2026-07-28 18:02:06.753335456 +0000 UTC m=+77.829897454"
Jul 28 18:02:06.770037 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:06.769985    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-zhfld" podStartSLOduration=33.680181688 podStartE2EDuration="35.769970413s" totalImagesPullingTime="2.089788725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:06.769065673 +0000 UTC m=+77.845627671" watchObservedRunningTime="2026-07-28 18:02:06.769970413 +0000 UTC m=+77.846532409"
Jul 28 18:02:07.125073 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:07.124977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:08.451789 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:08.451733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:02:08.453968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:08.453949    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10f0792e-73a1-45ed-93fd-508e7885060d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5nxds\" (UID: \"10f0792e-73a1-45ed-93fd-508e7885060d\") " pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:02:08.466992 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:08.466971    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4jjmm\""
Jul 28 18:02:08.474457 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:08.474441    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5nxds"
Jul 28 18:02:08.597753 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:08.597722    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5nxds"]
Jul 28 18:02:08.600710 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:02:08.600677    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10f0792e_73a1_45ed_93fd_508e7885060d.slice/crio-0e03c051af70fdd3a1a4e46e74d57c8ba32156b3af681a1d4316b625c8843452 WatchSource:0}: Error finding container 0e03c051af70fdd3a1a4e46e74d57c8ba32156b3af681a1d4316b625c8843452: Status 404 returned error can't find the container with id 0e03c051af70fdd3a1a4e46e74d57c8ba32156b3af681a1d4316b625c8843452
Jul 28 18:02:08.738605 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:08.738527    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5nxds" event={"ID":"10f0792e-73a1-45ed-93fd-508e7885060d","Type":"ContainerStarted","Data":"0e03c051af70fdd3a1a4e46e74d57c8ba32156b3af681a1d4316b625c8843452"}
Jul 28 18:02:10.268656 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.268606    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:10.276067 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.276031    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv"]
Jul 28 18:02:10.278228 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.278211    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv"
Jul 28 18:02:10.281544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.281521    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 18:02:10.284963 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.284943    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6sc6m\""
Jul 28 18:02:10.295521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.295497    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv"]
Jul 28 18:02:10.360704 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.360644    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:10.367909 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.367875    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b50cec69-24be-4ce0-b23d-89ced53eac8a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-n8ccv\" (UID: \"b50cec69-24be-4ce0-b23d-89ced53eac8a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv"
Jul 28 18:02:10.468521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.468436    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b50cec69-24be-4ce0-b23d-89ced53eac8a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-n8ccv\" (UID: \"b50cec69-24be-4ce0-b23d-89ced53eac8a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv"
Jul 28 18:02:10.471106 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.471079    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b50cec69-24be-4ce0-b23d-89ced53eac8a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-n8ccv\" (UID: \"b50cec69-24be-4ce0-b23d-89ced53eac8a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv"
Jul 28 18:02:10.588305 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.588260    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv"
Jul 28 18:02:10.725571 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:10.725493    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv"]
Jul 28 18:02:11.419416 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:11.419364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:11.519923 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:11.519865    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:11.712349 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:11.712311    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ftg8z"
Jul 28 18:02:12.994848 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:02:12.994796    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50cec69_24be_4ce0_b23d_89ced53eac8a.slice/crio-c0f9385e548c58b84c2e865f985f1185e447e7821b24f27592a5a177db623c56 WatchSource:0}: Error finding container c0f9385e548c58b84c2e865f985f1185e447e7821b24f27592a5a177db623c56: Status 404 returned error can't find the container with id c0f9385e548c58b84c2e865f985f1185e447e7821b24f27592a5a177db623c56
Jul 28 18:02:13.760610 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:13.760572    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5nxds" event={"ID":"10f0792e-73a1-45ed-93fd-508e7885060d","Type":"ContainerStarted","Data":"ae689aa0e58ccc1c1b3bcf4985e06090a49cb7f93166bcd7121c05006c086e9e"}
Jul 28 18:02:13.761847 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:13.761803    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv" event={"ID":"b50cec69-24be-4ce0-b23d-89ced53eac8a","Type":"ContainerStarted","Data":"c0f9385e548c58b84c2e865f985f1185e447e7821b24f27592a5a177db623c56"}
Jul 28 18:02:14.677906 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:14.677870    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:14.767396 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:14.767360    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv" event={"ID":"b50cec69-24be-4ce0-b23d-89ced53eac8a","Type":"ContainerStarted","Data":"22acf68c44df18686df15356c224f6ec3283404095985a359f1cf40c931bdc4e"}
Jul 28 18:02:14.767584 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:14.767568    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv"
Jul 28 18:02:14.775865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:14.775833    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv"
Jul 28 18:02:14.777481 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:14.777457    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:14.786044 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:14.785990    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n8ccv" podStartSLOduration=3.922418596 podStartE2EDuration="4.785971854s" totalImagesPullingTime="863.553258ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:14.785693573 +0000 UTC m=+85.862255626" watchObservedRunningTime="2026-07-28 18:02:14.785971854 +0000 UTC m=+85.862533852"
Jul 28 18:02:15.400173 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.400144    2583 scope.go:117] "RemoveContainer" containerID="49a45c39317c21dbcf2f2c58a0320c064deb8bc413f05c13706e75726be2036f"
Jul 28 18:02:15.400372 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:02:15.400341    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-t7pms_openshift-console-operator(abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6)\"" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" podUID="abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6"
Jul 28 18:02:15.621234 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.621196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:15.661333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.661226    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"]
Jul 28 18:02:15.674282 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.674250    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"]
Jul 28 18:02:15.674492 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.674468    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.677453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.677416    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-498z2\""
Jul 28 18:02:15.679123 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.679097    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 28 18:02:15.679505 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.679126    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 18:02:15.679505 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.679176    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 28 18:02:15.699346 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.699315    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:15.702212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.702187    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dkkq\" (UniqueName: \"kubernetes.io/projected/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-kube-api-access-9dkkq\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.702363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.702229    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.702363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.702323    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.702475 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.702380    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.803456 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.803403    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dkkq\" (UniqueName: \"kubernetes.io/projected/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-kube-api-access-9dkkq\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.803637 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.803473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.803637 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:02:15.803598    2583 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 28 18:02:15.803637 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.803625    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.803804 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:02:15.803667    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-prometheus-operator-tls podName:f9a3bc76-128c-4c0c-b4ab-1bf171726e0d nodeName:}" failed. No retries permitted until 2026-07-28 18:02:16.303647764 +0000 UTC m=+87.380209739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-dhpjg" (UID: "f9a3bc76-128c-4c0c-b4ab-1bf171726e0d") : secret "prometheus-operator-tls" not found
Jul 28 18:02:15.803804 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.803742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.804483 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.804461    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.806284 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.806259    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:15.813248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:15.813222    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dkkq\" (UniqueName: \"kubernetes.io/projected/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-kube-api-access-9dkkq\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:16.209314 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:16.209263    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:16.307517 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:16.307478    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:16.310716 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:16.310685    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9a3bc76-128c-4c0c-b4ab-1bf171726e0d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-dhpjg\" (UID: \"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:16.565069 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:16.564980    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:16.586860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:16.586808    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"
Jul 28 18:02:16.620373 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:16.620343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:16.911322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:16.911292    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-dhpjg"]
Jul 28 18:02:16.911941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:16.911919    2583 patch_prober.go:28] interesting pod/image-registry-5b567bb8ff-8p6pr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 18:02:16.912015 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:16.911963    2583 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr" podUID="2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 18:02:16.914268 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:02:16.914240    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9a3bc76_128c_4c0c_b4ab_1bf171726e0d.slice/crio-eb1c8cd1162e84bcebf23d01e0d4847dabb731b287480ae3385881a3c2141dba WatchSource:0}: Error finding container eb1c8cd1162e84bcebf23d01e0d4847dabb731b287480ae3385881a3c2141dba: Status 404 returned error can't find the container with id eb1c8cd1162e84bcebf23d01e0d4847dabb731b287480ae3385881a3c2141dba
Jul 28 18:02:17.782621 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:17.782580    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5nxds" event={"ID":"10f0792e-73a1-45ed-93fd-508e7885060d","Type":"ContainerStarted","Data":"c7373b16808d4078812baa7c0c7948dfb89f09bf392ace0a35c13ceed2659d4a"}
Jul 28 18:02:17.784142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:17.784111    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg" event={"ID":"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d","Type":"ContainerStarted","Data":"eb1c8cd1162e84bcebf23d01e0d4847dabb731b287480ae3385881a3c2141dba"}
Jul 28 18:02:18.383271 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:18.383178    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:18.515332 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:18.515268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:18.790353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:18.790315    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg" event={"ID":"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d","Type":"ContainerStarted","Data":"2e2e1336bf26cdba0c0ba63abcd277b07d0d92edb0b39d2547db38182d39d9e2"}
Jul 28 18:02:18.790542 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:18.790380    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg" event={"ID":"f9a3bc76-128c-4c0c-b4ab-1bf171726e0d","Type":"ContainerStarted","Data":"7d01ce885605301771c1210d5dce29cf868a548cd83374a96b63f7df3dc39fca"}
Jul 28 18:02:18.813082 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:18.813020    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-dhpjg" podStartSLOduration=2.620392513 podStartE2EDuration="3.813001771s" totalImagesPullingTime="1.192609258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:18.812296782 +0000 UTC m=+89.888858779" watchObservedRunningTime="2026-07-28 18:02:18.813001771 +0000 UTC m=+89.889563771"
Jul 28 18:02:19.700333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:19.700292    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:02:19.921467 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:19.921430    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:20.022049 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:20.021961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:20.761844 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:20.761773    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:20.802249 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:20.802215    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5nxds" event={"ID":"10f0792e-73a1-45ed-93fd-508e7885060d","Type":"ContainerStarted","Data":"44e3d9ae7be978089daa297b8fe01ce5c7b2e440cb4a309a204c93d63f39c581"}
Jul 28 18:02:20.822210 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:20.822163    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5nxds" podStartSLOduration=32.958866829 podStartE2EDuration="44.822148096s" totalImagesPullingTime="11.863281267s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:01:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:20.821674326 +0000 UTC m=+91.898236327" watchObservedRunningTime="2026-07-28 18:02:20.822148096 +0000 UTC m=+91.898710092"
Jul 28 18:02:20.861272 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:20.861241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:20.983794 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:20.983766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:21.026311 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.026282    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5q6wc"]
Jul 28 18:02:21.029425 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.029405    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.032644 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.032621    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-r5f9p\""
Jul 28 18:02:21.032961 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.032943    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 18:02:21.033052 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.032961    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 18:02:21.033052 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.032989    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 18:02:21.038656 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.038633    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca29f8de-1c36-49da-960e-cf14ae3c261e-root\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.038732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.038663    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-tls\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.038732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.038683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-wtmp\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.038732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.038702    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-textfile\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.038855 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.038792    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h4ss\" (UniqueName: \"kubernetes.io/projected/ca29f8de-1c36-49da-960e-cf14ae3c261e-kube-api-access-9h4ss\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.038891 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.038862    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca29f8de-1c36-49da-960e-cf14ae3c261e-sys\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.038921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.038890    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.038921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.038914    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca29f8de-1c36-49da-960e-cf14ae3c261e-metrics-client-ca\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.038984 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.038931    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-accelerators-collector-config\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139365    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca29f8de-1c36-49da-960e-cf14ae3c261e-sys\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139564 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139413    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139564 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139451    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca29f8de-1c36-49da-960e-cf14ae3c261e-metrics-client-ca\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139564 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139480    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-accelerators-collector-config\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139564 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139486    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca29f8de-1c36-49da-960e-cf14ae3c261e-sys\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139749 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139612    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca29f8de-1c36-49da-960e-cf14ae3c261e-root\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139749 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139642    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-tls\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139749 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139678    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-wtmp\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139749 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139707    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-textfile\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139970 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139752    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h4ss\" (UniqueName: \"kubernetes.io/projected/ca29f8de-1c36-49da-960e-cf14ae3c261e-kube-api-access-9h4ss\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.139970 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:02:21.139828    2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 18:02:21.139970 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:02:21.139887    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-tls podName:ca29f8de-1c36-49da-960e-cf14ae3c261e nodeName:}" failed. No retries permitted until 2026-07-28 18:02:21.639867047 +0000 UTC m=+92.716429040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-tls") pod "node-exporter-5q6wc" (UID: "ca29f8de-1c36-49da-960e-cf14ae3c261e") : secret "node-exporter-tls" not found
Jul 28 18:02:21.140122 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.139990    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-wtmp\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.140122 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.140097    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca29f8de-1c36-49da-960e-cf14ae3c261e-root\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.140214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.140143    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-accelerators-collector-config\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.140334 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.140306    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-textfile\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.140692 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.140668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:21.140791 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.140740    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca29f8de-1c36-49da-960e-cf14ae3c261e-metrics-client-ca\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.141985 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.141964    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.155474 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.155452    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h4ss\" (UniqueName: \"kubernetes.io/projected/ca29f8de-1c36-49da-960e-cf14ae3c261e-kube-api-access-9h4ss\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.644057 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.644021    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-tls\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.646267 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.646237    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca29f8de-1c36-49da-960e-cf14ae3c261e-node-exporter-tls\") pod \"node-exporter-5q6wc\" (UID: \"ca29f8de-1c36-49da-960e-cf14ae3c261e\") " pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.938231 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:21.938143    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5q6wc"
Jul 28 18:02:21.950967 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:02:21.950936    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca29f8de_1c36_49da_960e_cf14ae3c261e.slice/crio-3f424471e4e282ce26ebab244e7eafb3c4e7c74e3672cf2575a62566d6eb4629 WatchSource:0}: Error finding container 3f424471e4e282ce26ebab244e7eafb3c4e7c74e3672cf2575a62566d6eb4629: Status 404 returned error can't find the container with id 3f424471e4e282ce26ebab244e7eafb3c4e7c74e3672cf2575a62566d6eb4629
Jul 28 18:02:22.093472 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.093444    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 18:02:22.098120 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.098097    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.100939 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.100913    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 18:02:22.101941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.101698    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 18:02:22.101941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.101711    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 18:02:22.101941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.101732    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 18:02:22.101941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.101762    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 18:02:22.101941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.101773    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sntvq\""
Jul 28 18:02:22.101941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.101840    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 18:02:22.101941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.101703    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 18:02:22.101941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.101774    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 18:02:22.101941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.101718    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 18:02:22.115897 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.115872    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 18:02:22.146714 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.146681    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.146882 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.146723    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.146882 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.146739    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.146882 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.146805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.146882 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.146851    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.146882 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.146870    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.147103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.146893    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.147103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.146916    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.147103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.146942    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.147103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.146960    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr4kt\" (UniqueName: \"kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-kube-api-access-sr4kt\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.147103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.147007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-web-config\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.147103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.147032    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.147103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.147058    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-out\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.167697 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.167665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:22.233886 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.233848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:22.247511 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247485    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.247511 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247511    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.247712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247535    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.247712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247551    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.247712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247570    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.247712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.247712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247617    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.247712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247641    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.247712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr4kt\" (UniqueName: \"kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-kube-api-access-sr4kt\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.248073 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247713    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-web-config\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.248073 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247741    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.248073 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247783    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-out\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.248073 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.247832    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.249347 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.249318    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.249801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.249773    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.250362 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.250334    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.251583 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.251552    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-out\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.251667 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.251558    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.251667 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.251638    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.251968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.251949    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.252300 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.252278    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-web-config\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.252558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.252539    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.252628 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.252560    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.252722 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.252703    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.253089 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.253068    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.255771 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.255748    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr4kt\" (UniqueName: \"kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-kube-api-access-sr4kt\") pod \"alertmanager-main-0\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.412607 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.412577    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:02:22.569671 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.569639    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 18:02:22.569943 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:02:22.569920    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ea4b8e9_fad9_4c26_a6ec_08d5995e6cf9.slice/crio-4836475b2cd58301d81f0a8c8890b07be004c7ab2adc910614abdac0c0a8521f WatchSource:0}: Error finding container 4836475b2cd58301d81f0a8c8890b07be004c7ab2adc910614abdac0c0a8521f: Status 404 returned error can't find the container with id 4836475b2cd58301d81f0a8c8890b07be004c7ab2adc910614abdac0c0a8521f
Jul 28 18:02:22.811097 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.811005    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerStarted","Data":"4836475b2cd58301d81f0a8c8890b07be004c7ab2adc910614abdac0c0a8521f"}
Jul 28 18:02:22.812281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:22.812250    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5q6wc" event={"ID":"ca29f8de-1c36-49da-960e-cf14ae3c261e","Type":"ContainerStarted","Data":"3f424471e4e282ce26ebab244e7eafb3c4e7c74e3672cf2575a62566d6eb4629"}
Jul 28 18:02:23.818605 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:23.818530    2583 generic.go:356] "Generic (PLEG): container finished" podID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerID="617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f" exitCode=0
Jul 28 18:02:23.818605 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:23.818583    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerDied","Data":"617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f"}
Jul 28 18:02:23.964997 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:23.964965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:24.040452 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:24.040422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:24.179645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:24.179616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:24.523726 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:24.523674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:24.824786 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:24.824703    2583 generic.go:356] "Generic (PLEG): container finished" podID="ca29f8de-1c36-49da-960e-cf14ae3c261e" containerID="263f12c67f61b733e105211a62d3bd3b3831cac936c02d602f2c8a820a248a37" exitCode=0
Jul 28 18:02:24.825232 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:24.824796    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5q6wc" event={"ID":"ca29f8de-1c36-49da-960e-cf14ae3c261e","Type":"ContainerDied","Data":"263f12c67f61b733e105211a62d3bd3b3831cac936c02d602f2c8a820a248a37"}
Jul 28 18:02:25.360444 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.360417    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-567db57b8d-d65mx"]
Jul 28 18:02:25.365032 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.365013    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.368277 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.368253    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 18:02:25.368389 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.368294    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 18:02:25.368389 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.368365    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-d45tc\""
Jul 28 18:02:25.368562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.368545    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 18:02:25.368644 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.368583    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-brcubhs4cjt1v\""
Jul 28 18:02:25.368718 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.368699    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 18:02:25.373365 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.373343    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-audit-log\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.373448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.373379    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-metrics-server-audit-profiles\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.373448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.373400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-client-ca-bundle\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.373560 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.373520    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqptx\" (UniqueName: \"kubernetes.io/projected/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-kube-api-access-dqptx\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.373598 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.373571    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-secret-metrics-server-tls\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.373635 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.373598    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.373635 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.373628    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-secret-metrics-server-client-certs\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.380599 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.380577    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-567db57b8d-d65mx"]
Jul 28 18:02:25.474258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.474233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqptx\" (UniqueName: \"kubernetes.io/projected/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-kube-api-access-dqptx\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.474366 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.474270    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-secret-metrics-server-tls\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.474366 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.474296    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.474366 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.474336    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-secret-metrics-server-client-certs\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.474519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.474370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-audit-log\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.474567 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.474509    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-metrics-server-audit-profiles\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.474624 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.474567    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-client-ca-bundle\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.474799 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.474763    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-audit-log\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.475088 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.475062    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.475351 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.475330    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-metrics-server-audit-profiles\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.476655 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.476630    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-secret-metrics-server-tls\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.476727 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.476704    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-client-ca-bundle\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.476762 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.476738    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-secret-metrics-server-client-certs\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.484382 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.484361    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqptx\" (UniqueName: \"kubernetes.io/projected/76fa890b-57e3-4aa7-b6b8-73e325fac8f2-kube-api-access-dqptx\") pod \"metrics-server-567db57b8d-d65mx\" (UID: \"76fa890b-57e3-4aa7-b6b8-73e325fac8f2\") " pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.674422 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.674369    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:25.766221 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.766197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:25.770130 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.770103    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"]
Jul 28 18:02:25.774756 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.774731    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"
Jul 28 18:02:25.777884 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.777854    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 18:02:25.777984 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.777930    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7tz84\""
Jul 28 18:02:25.787134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.784425    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"]
Jul 28 18:02:25.819828 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.819784    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-567db57b8d-d65mx"]
Jul 28 18:02:25.822002 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:02:25.821976    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76fa890b_57e3_4aa7_b6b8_73e325fac8f2.slice/crio-b28885b0d5fd648d84ccecaa92e9ead2d92aad5bae3f1db1c5ad5a5e4030ed11 WatchSource:0}: Error finding container b28885b0d5fd648d84ccecaa92e9ead2d92aad5bae3f1db1c5ad5a5e4030ed11: Status 404 returned error can't find the container with id b28885b0d5fd648d84ccecaa92e9ead2d92aad5bae3f1db1c5ad5a5e4030ed11
Jul 28 18:02:25.830207 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.830178    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-567db57b8d-d65mx" event={"ID":"76fa890b-57e3-4aa7-b6b8-73e325fac8f2","Type":"ContainerStarted","Data":"b28885b0d5fd648d84ccecaa92e9ead2d92aad5bae3f1db1c5ad5a5e4030ed11"}
Jul 28 18:02:25.833118 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.833099    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerStarted","Data":"753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301"}
Jul 28 18:02:25.833201 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.833146    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerStarted","Data":"4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346"}
Jul 28 18:02:25.833201 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.833171    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerStarted","Data":"304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548"}
Jul 28 18:02:25.833201 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.833194    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerStarted","Data":"b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35"}
Jul 28 18:02:25.833320 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.833216    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerStarted","Data":"6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5"}
Jul 28 18:02:25.834934 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.834897    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5q6wc" event={"ID":"ca29f8de-1c36-49da-960e-cf14ae3c261e","Type":"ContainerStarted","Data":"6e75d2dade8e5084086613ffb9b5f6046c0dbf5e3c0240652269f97cb9aeedd4"}
Jul 28 18:02:25.835020 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.834943    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5q6wc" event={"ID":"ca29f8de-1c36-49da-960e-cf14ae3c261e","Type":"ContainerStarted","Data":"6c833832848924b1a1dc550fbf12bd01c9b84ea356b56bbdc1281e6574ce5bb6"}
Jul 28 18:02:25.855970 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.855930    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5q6wc" podStartSLOduration=2.724972944 podStartE2EDuration="4.855915849s" totalImagesPullingTime="2.067577201s" totalInitContainerRuntime="63.365704ms" isStatefulPod=true podCreationTimestamp="2026-07-28 18:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:25.854733283 +0000 UTC m=+96.931295274" watchObservedRunningTime="2026-07-28 18:02:25.855915849 +0000 UTC m=+96.932477875"
Jul 28 18:02:25.877382 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.877353    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhnsl\" (UniqueName: \"kubernetes.io/projected/0a6460a8-9d08-4c3a-98aa-726704e52594-kube-api-access-hhnsl\") pod \"monitoring-plugin-68b7fc9574-bvfmg\" (UID: \"0a6460a8-9d08-4c3a-98aa-726704e52594\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"
Jul 28 18:02:25.877507 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.877397    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0a6460a8-9d08-4c3a-98aa-726704e52594-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bvfmg\" (UID: \"0a6460a8-9d08-4c3a-98aa-726704e52594\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"
Jul 28 18:02:25.972629 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.972582    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:25.978495 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.978470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhnsl\" (UniqueName: \"kubernetes.io/projected/0a6460a8-9d08-4c3a-98aa-726704e52594-kube-api-access-hhnsl\") pod \"monitoring-plugin-68b7fc9574-bvfmg\" (UID: \"0a6460a8-9d08-4c3a-98aa-726704e52594\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"
Jul 28 18:02:25.978636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.978509    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0a6460a8-9d08-4c3a-98aa-726704e52594-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bvfmg\" (UID: \"0a6460a8-9d08-4c3a-98aa-726704e52594\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"
Jul 28 18:02:25.981312 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.981284    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0a6460a8-9d08-4c3a-98aa-726704e52594-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bvfmg\" (UID: \"0a6460a8-9d08-4c3a-98aa-726704e52594\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"
Jul 28 18:02:25.987510 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:25.987481    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhnsl\" (UniqueName: \"kubernetes.io/projected/0a6460a8-9d08-4c3a-98aa-726704e52594-kube-api-access-hhnsl\") pod \"monitoring-plugin-68b7fc9574-bvfmg\" (UID: \"0a6460a8-9d08-4c3a-98aa-726704e52594\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"
Jul 28 18:02:26.087228 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:26.087141    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"
Jul 28 18:02:26.318190 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:26.318160    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"]
Jul 28 18:02:26.321699 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:02:26.321667    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6460a8_9d08_4c3a_98aa_726704e52594.slice/crio-e2d85cdd12439f82aaf6da395d2512034c1dcc18ebb12016cada762883453631 WatchSource:0}: Error finding container e2d85cdd12439f82aaf6da395d2512034c1dcc18ebb12016cada762883453631: Status 404 returned error can't find the container with id e2d85cdd12439f82aaf6da395d2512034c1dcc18ebb12016cada762883453631
Jul 28 18:02:26.400060 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:26.399986    2583 scope.go:117] "RemoveContainer" containerID="49a45c39317c21dbcf2f2c58a0320c064deb8bc413f05c13706e75726be2036f"
Jul 28 18:02:26.844683 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:26.844473    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerStarted","Data":"da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed"}
Jul 28 18:02:26.846589 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:26.846556    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:02:26.846721 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:26.846693    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" event={"ID":"abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6","Type":"ContainerStarted","Data":"b2520765dd32ad38eb91c9fe2e90fe05abccd223798bab43e6e0c4f4f46138cc"}
Jul 28 18:02:26.847053 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:26.847021    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:02:26.850037 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:26.850005    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg" event={"ID":"0a6460a8-9d08-4c3a-98aa-726704e52594","Type":"ContainerStarted","Data":"e2d85cdd12439f82aaf6da395d2512034c1dcc18ebb12016cada762883453631"}
Jul 28 18:02:26.878834 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:26.878527    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.7914026870000002 podStartE2EDuration="4.878504516s" totalImagesPullingTime="3.068482113s" totalInitContainerRuntime="18.619716ms" isStatefulPod=false podCreationTimestamp="2026-07-28 18:02:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:26.876061778 +0000 UTC m=+97.952623832" watchObservedRunningTime="2026-07-28 18:02:26.878504516 +0000 UTC m=+97.955066492"
Jul 28 18:02:26.895219 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:26.895167    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms" podStartSLOduration=52.892013802 podStartE2EDuration="55.895151736s" totalImagesPullingTime="3.003137934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:26.894193132 +0000 UTC m=+97.970755130" watchObservedRunningTime="2026-07-28 18:02:26.895151736 +0000 UTC m=+97.971713733"
Jul 28 18:02:27.235632 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.235585    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:02:27.239408 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.239175    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:27.240288 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.240263    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.243597 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.243360    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 18:02:27.244082 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.244063    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 18:02:27.245415 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.245396    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 18:02:27.249125 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.245504    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3ur2gh26chm9q\""
Jul 28 18:02:27.249879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.245560    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 18:02:27.249879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.245597    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 18:02:27.249879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.245741    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 18:02:27.249879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.246158    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 18:02:27.249879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.246188    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 18:02:27.249879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.246228    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 18:02:27.249879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.246688    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 18:02:27.249879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.247169    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-d5dll\""
Jul 28 18:02:27.249879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.247281    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 18:02:27.249879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.247360    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 18:02:27.252982 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.252902    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 18:02:27.257958 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.257938    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:02:27.258711 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.258692    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 18:02:27.298968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.298867    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.298968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.298921    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.298968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.298958    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.298968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.298978    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.298968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.298996    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299385 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299018    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299385 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299036    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299385 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299055    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-web-config\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299385 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299082    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299385 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299121    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299385 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299238    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299385 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299315    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-config-out\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299385 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299347    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299385 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299379    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-config\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299841 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299404    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299841 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299427    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299841 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299455    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299841 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299479    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrtcd\" (UniqueName: \"kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-kube-api-access-zrtcd\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299841 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299513    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.299841 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.299543    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400103    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400329 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400157    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400329 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400193    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400329 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400232    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-config-out\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400329 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400260    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400329 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400287    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-config\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400329 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400368    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrtcd\" (UniqueName: \"kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-kube-api-access-zrtcd\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400423    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400453    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400525    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400556    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.400612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400596    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.401013 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400633    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.401013 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.401013 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400695    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.401013 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.400721    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-web-config\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.401469 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.401438    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.402348 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.402052    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.402348 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.402318    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.406258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.403709    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.406258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.404264    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.406258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.404887    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.406258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.405353    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.406258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.405720    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-web-config\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.406258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.406213    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.407366 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.407143    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.408225 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.407806    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-config-out\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.410248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.408905    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.411493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.411319    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.411493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.411427    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrtcd\" (UniqueName: \"kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-kube-api-access-zrtcd\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.411727 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.411629    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.411727 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.411719    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-config\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.411849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.411735    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.412293 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.412251    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.413846 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.413810    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.415081 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.415042    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.494460 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.494378    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-t7pms"
Jul 28 18:02:27.558402 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.558346    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:27.697856 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.697792    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:27.762715 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.762639    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:27.863039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.863008    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:27.950357 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.950326    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:02:27.953463 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:02:27.953439    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bcb9662_1721_44c5_a841_0156bff00d32.slice/crio-829f35dca00507fff27b7fed0afb95c32566e7ca19432e8906defa9a1e593733 WatchSource:0}: Error finding container 829f35dca00507fff27b7fed0afb95c32566e7ca19432e8906defa9a1e593733: Status 404 returned error can't find the container with id 829f35dca00507fff27b7fed0afb95c32566e7ca19432e8906defa9a1e593733
Jul 28 18:02:27.984313 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:27.984286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:28.179039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.179003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:28.279647 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.279613    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:28.379718 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.379622    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:28.446300 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.446250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:28.866873 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.866838    2583 generic.go:356] "Generic (PLEG): container finished" podID="0bcb9662-1721-44c5-a841-0156bff00d32" containerID="ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c" exitCode=0
Jul 28 18:02:28.867322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.866911    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerDied","Data":"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"}
Jul 28 18:02:28.867322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.867002    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerStarted","Data":"829f35dca00507fff27b7fed0afb95c32566e7ca19432e8906defa9a1e593733"}
Jul 28 18:02:28.868605 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.868582    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-567db57b8d-d65mx" event={"ID":"76fa890b-57e3-4aa7-b6b8-73e325fac8f2","Type":"ContainerStarted","Data":"71cd1c4c0782af001f8c705fbb10c2ad6405899e83f8c70b8c0fa1f996108896"}
Jul 28 18:02:28.870139 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.870120    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg" event={"ID":"0a6460a8-9d08-4c3a-98aa-726704e52594","Type":"ContainerStarted","Data":"563a2a2ad6cd58484b2cb7f6bb6c8bb509a074a78af4cd59555f9eec51faec78"}
Jul 28 18:02:28.912316 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.912264    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg" podStartSLOduration=2.43742534 podStartE2EDuration="3.912249471s" totalImagesPullingTime="1.474824131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:02:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:28.911080496 +0000 UTC m=+99.987642494" watchObservedRunningTime="2026-07-28 18:02:28.912249471 +0000 UTC m=+99.988811467"
Jul 28 18:02:28.931045 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.930991    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-567db57b8d-d65mx" podStartSLOduration=1.958658576 podStartE2EDuration="3.930974837s" totalImagesPullingTime="1.972316261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:02:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:28.929825466 +0000 UTC m=+100.006387455" watchObservedRunningTime="2026-07-28 18:02:28.930974837 +0000 UTC m=+100.007536833"
Jul 28 18:02:28.995658 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:28.995628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:29.056456 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:29.056420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:29.199719 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:29.199692    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:29.309680 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:29.309635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:29.411364 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:29.411330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:29.529724 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:29.529616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:29.629422 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:29.629383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:29.696344 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:29.696251    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:29.875400 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:29.875318    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"
Jul 28 18:02:29.880772 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:29.880746    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bvfmg"
Jul 28 18:02:29.957511 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:29.957472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:30.023521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:30.023484    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:31.392880 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:31.392720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:31.466780 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:31.466738    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:32.474753 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:32.474719    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b567bb8ff-8p6pr"]
Jul 28 18:02:32.577590 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:32.577511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:32.678693 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:32.678654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:32.712262 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:32.712233    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kbqwj"
Jul 28 18:02:32.891345 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:32.891266    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerStarted","Data":"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"}
Jul 28 18:02:32.891345 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:32.891341    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerStarted","Data":"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"}
Jul 28 18:02:34.902711 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:34.902684    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerStarted","Data":"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"}
Jul 28 18:02:35.146054 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:35.146026    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:35.248506 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:35.248406    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:35.910428 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:35.910385    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerStarted","Data":"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"}
Jul 28 18:02:35.910838 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:35.910463    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerStarted","Data":"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"}
Jul 28 18:02:35.910838 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:35.910503    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerStarted","Data":"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"}
Jul 28 18:02:35.943253 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:35.943187    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.364649374 podStartE2EDuration="8.943166413s" totalImagesPullingTime="5.561877889s" totalInitContainerRuntime="16.63915ms" isStatefulPod=false podCreationTimestamp="2026-07-28 18:02:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:02:35.940907933 +0000 UTC m=+107.017469929" watchObservedRunningTime="2026-07-28 18:02:35.943166413 +0000 UTC m=+107.019728414"
Jul 28 18:02:37.558674 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:37.558645    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:38.609361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:38.609328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:38.706609 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:38.706577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:38.807221 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:38.807184    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:38.861333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:38.861257    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:44.795278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:44.795232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:44.896267 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:44.896231    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:44.997225 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:44.997186    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:45.062663 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:45.062560    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:45.675380 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:45.675345    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:45.675582 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:45.675415    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:02:48.989687 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:48.989659    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:49.047914 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:49.047883    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:56.399978 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:56.399942    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:56.461328 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:56.461291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:02:57.495621 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.495582    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr" podUID="2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" containerName="registry" containerID="cri-o://c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb" gracePeriod=30
Jul 28 18:02:57.738575 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.738553    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:02:57.886067 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.885988    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-ca-trust-extracted\") pod \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") "
Jul 28 18:02:57.886067 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.886029    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-certificates\") pod \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") "
Jul 28 18:02:57.886067 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.886065    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-bound-sa-token\") pod \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") "
Jul 28 18:02:57.886338 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.886099    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-image-registry-private-configuration\") pod \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") "
Jul 28 18:02:57.886338 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.886145    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") pod \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") "
Jul 28 18:02:57.886338 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.886177    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s5nv\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-kube-api-access-7s5nv\") pod \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") "
Jul 28 18:02:57.886338 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.886212    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-trusted-ca\") pod \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") "
Jul 28 18:02:57.886338 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.886240    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-installation-pull-secrets\") pod \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\" (UID: \"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8\") "
Jul 28 18:02:57.886615 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.886449    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:02:57.886809 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.886779    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:02:57.888716 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.888668    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:02:57.888839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.888761    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:02:57.889043 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.889021    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-kube-api-access-7s5nv" (OuterVolumeSpecName: "kube-api-access-7s5nv") pod "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8"). InnerVolumeSpecName "kube-api-access-7s5nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:02:57.889100 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.889044    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:02:57.889157 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.889129    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:02:57.894499 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.894472    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" (UID: "2e61e8b3-e5cd-4e79-ab56-47a81061fdc8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:02:57.987143 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.987113    2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-tls\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:02:57.987143 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.987139    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s5nv\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-kube-api-access-7s5nv\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:02:57.987143 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.987148    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-trusted-ca\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:02:57.987332 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.987157    2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-installation-pull-secrets\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:02:57.987332 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.987176    2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-ca-trust-extracted\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:02:57.987332 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.987186    2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-registry-certificates\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:02:57.987332 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.987195    2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-bound-sa-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:02:57.987332 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.987204    2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8-image-registry-private-configuration\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:02:57.996365 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.996339    2583 generic.go:356] "Generic (PLEG): container finished" podID="2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" containerID="c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb" exitCode=0
Jul 28 18:02:57.996503 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.996389    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr" event={"ID":"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8","Type":"ContainerDied","Data":"c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb"}
Jul 28 18:02:57.996503 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.996394    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr"
Jul 28 18:02:57.996503 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.996432    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b567bb8ff-8p6pr" event={"ID":"2e61e8b3-e5cd-4e79-ab56-47a81061fdc8","Type":"ContainerDied","Data":"d9586c69792d0b1435aedc71ad75c17d367203c26ca3dd1034e7bf29b9b389a8"}
Jul 28 18:02:57.996503 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:57.996452    2583 scope.go:117] "RemoveContainer" containerID="c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb"
Jul 28 18:02:58.005610 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:58.005593    2583 scope.go:117] "RemoveContainer" containerID="c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb"
Jul 28 18:02:58.005918 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:02:58.005900    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb\": container with ID starting with c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb not found: ID does not exist" containerID="c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb"
Jul 28 18:02:58.005991 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:58.005924    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb"} err="failed to get container status \"c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb\": rpc error: code = NotFound desc = could not find container \"c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb\": container with ID starting with c40d99995aaae2bcf2667a750ba04ac39d2da6d63cf9c8c091f4a0af3b0130eb not found: ID does not exist"
Jul 28 18:02:58.019525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:58.019487    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b567bb8ff-8p6pr"]
Jul 28 18:02:58.028312 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:58.028286    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b567bb8ff-8p6pr"]
Jul 28 18:02:59.404997 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:59.404956    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" path="/var/lib/kubelet/pods/2e61e8b3-e5cd-4e79-ab56-47a81061fdc8/volumes"
Jul 28 18:02:59.958224 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:02:59.958194    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:00.027273 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:00.027241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:01.191348 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:01.191310    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:02.004424 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:02.004358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:02.064829 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:02.064776    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:02.198509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:02.198476    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:02.259077 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:02.258986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:05.680571 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:05.680536    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:03:05.684304 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:05.684283    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-567db57b8d-d65mx"
Jul 28 18:03:07.033223 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:07.033192    2583 generic.go:356] "Generic (PLEG): container finished" podID="bb04a323-a317-43c8-b604-d817792bdf73" containerID="dce4311b7f8fcb43ea6a1a9f5d092d950d9cd66fc116c22fded4e10c22298612" exitCode=0
Jul 28 18:03:07.033649 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:07.033374    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-7p6xv" event={"ID":"bb04a323-a317-43c8-b604-d817792bdf73","Type":"ContainerDied","Data":"dce4311b7f8fcb43ea6a1a9f5d092d950d9cd66fc116c22fded4e10c22298612"}
Jul 28 18:03:07.034175 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:07.034154    2583 scope.go:117] "RemoveContainer" containerID="dce4311b7f8fcb43ea6a1a9f5d092d950d9cd66fc116c22fded4e10c22298612"
Jul 28 18:03:08.038730 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:08.038692    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-7p6xv" event={"ID":"bb04a323-a317-43c8-b604-d817792bdf73","Type":"ContainerStarted","Data":"f950bb4850aafa3c432cb814c90ed69aea6a99039bedd78525142be1467d6fe7"}
Jul 28 18:03:27.559065 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:27.559036    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:27.573530 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:27.573504    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:28.136113 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:28.136087    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:40.273965 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:40.273921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:40.778710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:40.778663    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:41.301878 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:41.301842    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 18:03:41.302277 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:41.302171    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy-metric" containerID="cri-o://753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301" gracePeriod=120
Jul 28 18:03:41.302277 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:41.302188    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy-web" containerID="cri-o://304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548" gracePeriod=120
Jul 28 18:03:41.302277 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:41.302207    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy" containerID="cri-o://4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346" gracePeriod=120
Jul 28 18:03:41.302277 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:41.302242    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="prom-label-proxy" containerID="cri-o://da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed" gracePeriod=120
Jul 28 18:03:41.302489 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:41.302276    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="config-reloader" containerID="cri-o://b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35" gracePeriod=120
Jul 28 18:03:41.302489 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:41.302159    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="alertmanager" containerID="cri-o://6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5" gracePeriod=120
Jul 28 18:03:42.177262 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.177230    2583 generic.go:356] "Generic (PLEG): container finished" podID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerID="da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed" exitCode=0
Jul 28 18:03:42.177262 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.177256    2583 generic.go:356] "Generic (PLEG): container finished" podID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerID="4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346" exitCode=0
Jul 28 18:03:42.177262 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.177265    2583 generic.go:356] "Generic (PLEG): container finished" podID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerID="b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35" exitCode=0
Jul 28 18:03:42.177262 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.177271    2583 generic.go:356] "Generic (PLEG): container finished" podID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerID="6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5" exitCode=0
Jul 28 18:03:42.177534 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.177300    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerDied","Data":"da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed"}
Jul 28 18:03:42.177534 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.177371    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerDied","Data":"4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346"}
Jul 28 18:03:42.177534 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.177398    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerDied","Data":"b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35"}
Jul 28 18:03:42.177534 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.177424    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerDied","Data":"6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5"}
Jul 28 18:03:42.547985 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.547960    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:42.645046 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645013    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-web\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645199 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645055    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-volume\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645199 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645091    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr4kt\" (UniqueName: \"kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-kube-api-access-sr4kt\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645199 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645111    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-out\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645199 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645137    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-cluster-tls-config\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645199 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645156    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-metrics-client-ca\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645199 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645176    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-trusted-ca-bundle\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645199 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645191    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-main-tls\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645207    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-web-config\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645269    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645305    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-main-db\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645347    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-tls-assets\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645373    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy\") pod \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\" (UID: \"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9\") "
Jul 28 18:03:42.645779 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645686    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:03:42.645779 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.645743    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:03:42.646222 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.646171    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:03:42.648211 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.648185    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-volume" (OuterVolumeSpecName: "config-volume") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:42.648575 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.648552    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-kube-api-access-sr4kt" (OuterVolumeSpecName: "kube-api-access-sr4kt") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "kube-api-access-sr4kt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:03:42.648998 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.648959    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-out" (OuterVolumeSpecName: "config-out") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:03:42.649453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.649292    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:42.649453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.649370    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:42.649453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.649428    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:42.649453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.649447    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:03:42.649700 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.649527    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:42.653151 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.653062    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:42.659463 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.659444    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-web-config" (OuterVolumeSpecName: "web-config") pod "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" (UID: "5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:42.746943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746860    2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-tls-assets\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.746943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746893    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.746943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746904    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.746943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746913    2583 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-volume\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.746943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746922    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr4kt\" (UniqueName: \"kubernetes.io/projected/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-kube-api-access-sr4kt\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.746943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746931    2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-config-out\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.746943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746939    2583 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-cluster-tls-config\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.746943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746947    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-metrics-client-ca\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.747259 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746956    2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.747259 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746964    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-main-tls\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.747259 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746972    2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-web-config\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.747259 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746981    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:42.747259 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:42.746990    2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9-alertmanager-main-db\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:43.184212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.184119    2583 generic.go:356] "Generic (PLEG): container finished" podID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerID="753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301" exitCode=0
Jul 28 18:03:43.184212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.184159    2583 generic.go:356] "Generic (PLEG): container finished" podID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerID="304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548" exitCode=0
Jul 28 18:03:43.184212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.184198    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerDied","Data":"753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301"}
Jul 28 18:03:43.184411 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.184229    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.184411 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.184269    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerDied","Data":"304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548"}
Jul 28 18:03:43.184411 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.184298    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9","Type":"ContainerDied","Data":"4836475b2cd58301d81f0a8c8890b07be004c7ab2adc910614abdac0c0a8521f"}
Jul 28 18:03:43.184411 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.184329    2583 scope.go:117] "RemoveContainer" containerID="da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed"
Jul 28 18:03:43.192524 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.192419    2583 scope.go:117] "RemoveContainer" containerID="753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301"
Jul 28 18:03:43.199447 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.199433    2583 scope.go:117] "RemoveContainer" containerID="4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346"
Jul 28 18:03:43.205763 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.205744    2583 scope.go:117] "RemoveContainer" containerID="304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548"
Jul 28 18:03:43.210581 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.210557    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 18:03:43.218378 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.218358    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 18:03:43.219722 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.219706    2583 scope.go:117] "RemoveContainer" containerID="b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35"
Jul 28 18:03:43.226869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.226844    2583 scope.go:117] "RemoveContainer" containerID="6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5"
Jul 28 18:03:43.234099 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.234082    2583 scope.go:117] "RemoveContainer" containerID="617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f"
Jul 28 18:03:43.241310 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.241263    2583 scope.go:117] "RemoveContainer" containerID="da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed"
Jul 28 18:03:43.241554 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:43.241524    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed\": container with ID starting with da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed not found: ID does not exist" containerID="da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed"
Jul 28 18:03:43.241613 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.241562    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed"} err="failed to get container status \"da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed\": rpc error: code = NotFound desc = could not find container \"da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed\": container with ID starting with da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed not found: ID does not exist"
Jul 28 18:03:43.241613 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.241579    2583 scope.go:117] "RemoveContainer" containerID="753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301"
Jul 28 18:03:43.241853 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:43.241837    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301\": container with ID starting with 753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301 not found: ID does not exist" containerID="753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301"
Jul 28 18:03:43.241904 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.241858    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301"} err="failed to get container status \"753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301\": rpc error: code = NotFound desc = could not find container \"753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301\": container with ID starting with 753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301 not found: ID does not exist"
Jul 28 18:03:43.241904 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.241872    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 18:03:43.241970 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.241877    2583 scope.go:117] "RemoveContainer" containerID="4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346"
Jul 28 18:03:43.242193 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:43.242174    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346\": container with ID starting with 4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346 not found: ID does not exist" containerID="4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242201    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="config-reloader"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242219    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="config-reloader"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242233    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="alertmanager"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242239    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="alertmanager"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242246    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy-web"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242252    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy-web"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242258    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy-metric"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242263    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy-metric"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242269    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="prom-label-proxy"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242274    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="prom-label-proxy"
Jul 28 18:03:43.242276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242197    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346"} err="failed to get container status \"4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346\": rpc error: code = NotFound desc = could not find container \"4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346\": container with ID starting with 4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346 not found: ID does not exist"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242289    2583 scope.go:117] "RemoveContainer" containerID="304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242281    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242340    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242354    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" containerName="registry"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242362    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" containerName="registry"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242377    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="init-config-reloader"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242386    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="init-config-reloader"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242474    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="config-reloader"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242486    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy-web"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242499    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy-metric"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242507    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e61e8b3-e5cd-4e79-ab56-47a81061fdc8" containerName="registry"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242515    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="alertmanager"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242520    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="prom-label-proxy"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242528    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" containerName="kube-rbac-proxy"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:43.242551    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548\": container with ID starting with 304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548 not found: ID does not exist" containerID="304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242570    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548"} err="failed to get container status \"304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548\": rpc error: code = NotFound desc = could not find container \"304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548\": container with ID starting with 304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548 not found: ID does not exist"
Jul 28 18:03:43.242638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242585    2583 scope.go:117] "RemoveContainer" containerID="b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35"
Jul 28 18:03:43.243292 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:43.242834    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35\": container with ID starting with b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35 not found: ID does not exist" containerID="b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35"
Jul 28 18:03:43.243292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242851    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35"} err="failed to get container status \"b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35\": rpc error: code = NotFound desc = could not find container \"b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35\": container with ID starting with b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35 not found: ID does not exist"
Jul 28 18:03:43.243292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.242865    2583 scope.go:117] "RemoveContainer" containerID="6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5"
Jul 28 18:03:43.243292 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:43.243094    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5\": container with ID starting with 6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5 not found: ID does not exist" containerID="6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5"
Jul 28 18:03:43.243292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.243120    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5"} err="failed to get container status \"6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5\": rpc error: code = NotFound desc = could not find container \"6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5\": container with ID starting with 6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5 not found: ID does not exist"
Jul 28 18:03:43.243292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.243135    2583 scope.go:117] "RemoveContainer" containerID="617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f"
Jul 28 18:03:43.243571 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:43.243368    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f\": container with ID starting with 617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f not found: ID does not exist" containerID="617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f"
Jul 28 18:03:43.243571 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.243393    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f"} err="failed to get container status \"617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f\": rpc error: code = NotFound desc = could not find container \"617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f\": container with ID starting with 617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f not found: ID does not exist"
Jul 28 18:03:43.243571 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.243406    2583 scope.go:117] "RemoveContainer" containerID="da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed"
Jul 28 18:03:43.243679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.243642    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed"} err="failed to get container status \"da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed\": rpc error: code = NotFound desc = could not find container \"da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed\": container with ID starting with da40077ca890dee6bfcbd8675c2ae7074663cc3cf2468c668a1c651c2b0a90ed not found: ID does not exist"
Jul 28 18:03:43.243679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.243657    2583 scope.go:117] "RemoveContainer" containerID="753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301"
Jul 28 18:03:43.243920 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.243875    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301"} err="failed to get container status \"753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301\": rpc error: code = NotFound desc = could not find container \"753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301\": container with ID starting with 753b92fe80cf18321cdf5534ec6cc928891987c73156445313290cee92f62301 not found: ID does not exist"
Jul 28 18:03:43.243920 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.243901    2583 scope.go:117] "RemoveContainer" containerID="4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346"
Jul 28 18:03:43.244189 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.244168    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346"} err="failed to get container status \"4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346\": rpc error: code = NotFound desc = could not find container \"4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346\": container with ID starting with 4787b49686c9644fe2493d4cd8cfc6e42b808fcea6c39d0c3842060303342346 not found: ID does not exist"
Jul 28 18:03:43.244189 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.244188    2583 scope.go:117] "RemoveContainer" containerID="304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548"
Jul 28 18:03:43.244434 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.244408    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548"} err="failed to get container status \"304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548\": rpc error: code = NotFound desc = could not find container \"304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548\": container with ID starting with 304bd361ea466e0980893a650b386573f5c443e4518b32497f6a442eb2d67548 not found: ID does not exist"
Jul 28 18:03:43.244516 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.244436    2583 scope.go:117] "RemoveContainer" containerID="b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35"
Jul 28 18:03:43.244672 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.244655    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35"} err="failed to get container status \"b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35\": rpc error: code = NotFound desc = could not find container \"b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35\": container with ID starting with b723b722de6b6f5aa2aeb21048511f57aa65cfe49dcb195315672fd4e315ae35 not found: ID does not exist"
Jul 28 18:03:43.244724 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.244673    2583 scope.go:117] "RemoveContainer" containerID="6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5"
Jul 28 18:03:43.244927 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.244896    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5"} err="failed to get container status \"6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5\": rpc error: code = NotFound desc = could not find container \"6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5\": container with ID starting with 6ff309c06e7527afd9899ee4f5b5aa885d63d31abf1e00963d56e0bab32fc9f5 not found: ID does not exist"
Jul 28 18:03:43.244927 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.244918    2583 scope.go:117] "RemoveContainer" containerID="617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f"
Jul 28 18:03:43.245134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.245117    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f"} err="failed to get container status \"617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f\": rpc error: code = NotFound desc = could not find container \"617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f\": container with ID starting with 617dfd15bd1dcff8ee33f70d909af6c7c6a5c0709b0fde5a284e91666993ce3f not found: ID does not exist"
Jul 28 18:03:43.247288 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.247273    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.250504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.250482    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 18:03:43.250504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.250496    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 18:03:43.250657 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.250486    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 18:03:43.250657 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.250543    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 18:03:43.250657 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.250486    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 18:03:43.250657 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.250482    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sntvq\""
Jul 28 18:03:43.250935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.250700    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 18:03:43.250935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.250722    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 18:03:43.250935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.250759    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 18:03:43.255090 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.255066    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 18:03:43.263304 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.263203    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 18:03:43.352245 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352209    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352245 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352248    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352288    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352379    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352406    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352426    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352583 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352477    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8g69\" (UniqueName: \"kubernetes.io/projected/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-kube-api-access-k8g69\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352583 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352495    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352583 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352515    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352583 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-config-out\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352774 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352583    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352774 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352622    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.352774 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.352639    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-web-config\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.404780 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.404743    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9" path="/var/lib/kubelet/pods/5ea4b8e9-fad9-4c26-a6ec-08d5995e6cf9/volumes"
Jul 28 18:03:43.453555 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.453521    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.453717 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.453562    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-config-out\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.453717 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.453594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.453717 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.453631    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.453717 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.453656    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-web-config\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.453960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.453811    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.453960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.453875    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.453960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.453931    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.454117 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.453991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.454117 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.454001    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.454117 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.454020    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.454117 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.454046    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.454117 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.454095    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8g69\" (UniqueName: \"kubernetes.io/projected/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-kube-api-access-k8g69\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.454365 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.454122    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.455362 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.455312    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.455716 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.455695    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.457009 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.456940    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.457256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.457231    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-web-config\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.457374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.457333    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.457474 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.457377    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.457474 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.457440    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-config-volume\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.457589 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.457571    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-config-out\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.457700 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.457676    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.458012 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.457995    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.458381 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.458361    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.463877 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.463858    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8g69\" (UniqueName: \"kubernetes.io/projected/b25b020d-dea5-4c28-b08a-a1c7db50a5ea-kube-api-access-k8g69\") pod \"alertmanager-main-0\" (UID: \"b25b020d-dea5-4c28-b08a-a1c7db50a5ea\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.557013 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.556970    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 18:03:43.708327 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:43.708244    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 18:03:43.712270 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:03:43.712239    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25b020d_dea5_4c28_b08a_a1c7db50a5ea.slice/crio-1d8dc364cf5e8bc2172f702e759757ff97cbcf4a9e5542a217e819789209c284 WatchSource:0}: Error finding container 1d8dc364cf5e8bc2172f702e759757ff97cbcf4a9e5542a217e819789209c284: Status 404 returned error can't find the container with id 1d8dc364cf5e8bc2172f702e759757ff97cbcf4a9e5542a217e819789209c284
Jul 28 18:03:44.190630 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:44.190598    2583 generic.go:356] "Generic (PLEG): container finished" podID="b25b020d-dea5-4c28-b08a-a1c7db50a5ea" containerID="31c7cd9bc5de378c0866b7d0cbfaea20b34b66678febfbe88b4debd0f9bc4aae" exitCode=0
Jul 28 18:03:44.190787 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:44.190655    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b25b020d-dea5-4c28-b08a-a1c7db50a5ea","Type":"ContainerDied","Data":"31c7cd9bc5de378c0866b7d0cbfaea20b34b66678febfbe88b4debd0f9bc4aae"}
Jul 28 18:03:44.190787 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:44.190725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b25b020d-dea5-4c28-b08a-a1c7db50a5ea","Type":"ContainerStarted","Data":"1d8dc364cf5e8bc2172f702e759757ff97cbcf4a9e5542a217e819789209c284"}
Jul 28 18:03:45.198609 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.198573    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b25b020d-dea5-4c28-b08a-a1c7db50a5ea","Type":"ContainerStarted","Data":"129a73eeffbe88299f66ea2550dbc2de77718833582c6542fdbb616eb7c83607"}
Jul 28 18:03:45.199016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.198635    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b25b020d-dea5-4c28-b08a-a1c7db50a5ea","Type":"ContainerStarted","Data":"de9fec1d9dd49c620861bb9644eaf2290b0efb6aaf35e2d2ca7955a7b64324f3"}
Jul 28 18:03:45.199016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.198664    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b25b020d-dea5-4c28-b08a-a1c7db50a5ea","Type":"ContainerStarted","Data":"ae01c2f333962b187f9bcdcf45590267bbf3196734a277a08ef64e8915032080"}
Jul 28 18:03:45.199016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.198688    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b25b020d-dea5-4c28-b08a-a1c7db50a5ea","Type":"ContainerStarted","Data":"215e4b5ed36253a7a1c33c9388418eae58d2504171e2aac441adfd12aae9d39e"}
Jul 28 18:03:45.199016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.198716    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b25b020d-dea5-4c28-b08a-a1c7db50a5ea","Type":"ContainerStarted","Data":"b30be2ac3f3a629dc61ed20cc943c21774d2f4c633f476d599e424b8b9eaf4e7"}
Jul 28 18:03:45.199016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.198739    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b25b020d-dea5-4c28-b08a-a1c7db50a5ea","Type":"ContainerStarted","Data":"82c2e49ff66b7a80e54ec6eda18d5bfc7e1e6bbefe9bcfeedccba14fce4fc924"}
Jul 28 18:03:45.231053 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.231002    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.216716186 podStartE2EDuration="2.230986238s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.270052ms" isStatefulPod=false podCreationTimestamp="2026-07-28 18:03:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:03:45.228842861 +0000 UTC m=+176.305404859" watchObservedRunningTime="2026-07-28 18:03:45.230986238 +0000 UTC m=+176.307548235"
Jul 28 18:03:45.658309 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.658221    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:03:45.658600 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.658570    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="prometheus" containerID="cri-o://c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f" gracePeriod=600
Jul 28 18:03:45.658710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.658606    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy" containerID="cri-o://c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40" gracePeriod=600
Jul 28 18:03:45.658710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.658632    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy-thanos" containerID="cri-o://993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6" gracePeriod=600
Jul 28 18:03:45.658710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.658653    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="config-reloader" containerID="cri-o://21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22" gracePeriod=600
Jul 28 18:03:45.658912 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.658640    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="thanos-sidecar" containerID="cri-o://58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2" gracePeriod=600
Jul 28 18:03:45.659558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.659017    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy-web" containerID="cri-o://4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c" gracePeriod=600
Jul 28 18:03:45.908579 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.908504    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:45.981545 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.981511    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-db\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.981735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.981568    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrtcd\" (UniqueName: \"kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-kube-api-access-zrtcd\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.981735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.981614    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-config-out\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.981735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.981675    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-0\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.981735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.981699    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-trusted-ca-bundle\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.981735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.981732    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-tls\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.981764    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-1\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.981791    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-tls-assets\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982447 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982420    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-serving-certs-ca-bundle\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982550 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982472    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:03:45.982550 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982487    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982651 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982563    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-web-config\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982651 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982591    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:03:45.982651 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982600    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982651 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982633    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-kube-rbac-proxy\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982663    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-metrics-client-ca\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982691    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-grpc-tls\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982728    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-2\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982754    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-kubelet-serving-ca-bundle\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982788    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-thanos-prometheus-http-client-file\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982828    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-metrics-client-certs\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982863    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-config\") pod \"0bcb9662-1721-44c5-a841-0156bff00d32\" (UID: \"0bcb9662-1721-44c5-a841-0156bff00d32\") "
Jul 28 18:03:45.982875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.982856    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:03:45.983259 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.983165    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:03:45.983891 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.983337    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984186    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984313    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984614    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-db\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984635    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984649    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984663    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984665    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984680    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984694    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984708    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984770    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-config-out" (OuterVolumeSpecName: "config-out") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:03:45.984929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.984808    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-kube-api-access-zrtcd" (OuterVolumeSpecName: "kube-api-access-zrtcd") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "kube-api-access-zrtcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:03:45.986214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.986192    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:03:45.986320 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.986208    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:45.986401 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.986374    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:45.986905 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.986869    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:45.987359 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.987324    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:45.987575 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.987549    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:45.987766 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.987731    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:45.987879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.987865    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:45.988179 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.988155    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-config" (OuterVolumeSpecName: "config") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:45.997317 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:45.997284    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-web-config" (OuterVolumeSpecName: "web-config") pod "0bcb9662-1721-44c5-a841-0156bff00d32" (UID: "0bcb9662-1721-44c5-a841-0156bff00d32"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:03:46.085905 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085873    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.085905 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085902    2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-tls-assets\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085915    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085924    2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-web-config\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085933    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085943    2583 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-kube-rbac-proxy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085953    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0bcb9662-1721-44c5-a841-0156bff00d32-configmap-metrics-client-ca\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085961    2583 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-grpc-tls\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085970    2583 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085978    2583 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-secret-metrics-client-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085986    2583 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/0bcb9662-1721-44c5-a841-0156bff00d32-config\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.085994    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrtcd\" (UniqueName: \"kubernetes.io/projected/0bcb9662-1721-44c5-a841-0156bff00d32-kube-api-access-zrtcd\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.086096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.086003    2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0bcb9662-1721-44c5-a841-0156bff00d32-config-out\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:03:46.205885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.205853    2583 generic.go:356] "Generic (PLEG): container finished" podID="0bcb9662-1721-44c5-a841-0156bff00d32" containerID="993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6" exitCode=0
Jul 28 18:03:46.205885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.205878    2583 generic.go:356] "Generic (PLEG): container finished" podID="0bcb9662-1721-44c5-a841-0156bff00d32" containerID="c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40" exitCode=0
Jul 28 18:03:46.205885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.205884    2583 generic.go:356] "Generic (PLEG): container finished" podID="0bcb9662-1721-44c5-a841-0156bff00d32" containerID="4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c" exitCode=0
Jul 28 18:03:46.205885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.205890    2583 generic.go:356] "Generic (PLEG): container finished" podID="0bcb9662-1721-44c5-a841-0156bff00d32" containerID="58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2" exitCode=0
Jul 28 18:03:46.205885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.205895    2583 generic.go:356] "Generic (PLEG): container finished" podID="0bcb9662-1721-44c5-a841-0156bff00d32" containerID="21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22" exitCode=0
Jul 28 18:03:46.206586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.205901    2583 generic.go:356] "Generic (PLEG): container finished" podID="0bcb9662-1721-44c5-a841-0156bff00d32" containerID="c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f" exitCode=0
Jul 28 18:03:46.206586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.205929    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.206586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.205944    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerDied","Data":"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"}
Jul 28 18:03:46.206586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.206038    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerDied","Data":"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"}
Jul 28 18:03:46.206586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.206080    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerDied","Data":"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"}
Jul 28 18:03:46.206586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.206099    2583 scope.go:117] "RemoveContainer" containerID="993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"
Jul 28 18:03:46.206586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.206126    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerDied","Data":"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"}
Jul 28 18:03:46.206586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.206170    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerDied","Data":"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"}
Jul 28 18:03:46.206586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.206211    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerDied","Data":"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"}
Jul 28 18:03:46.206586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.206247    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0bcb9662-1721-44c5-a841-0156bff00d32","Type":"ContainerDied","Data":"829f35dca00507fff27b7fed0afb95c32566e7ca19432e8906defa9a1e593733"}
Jul 28 18:03:46.213916 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.213799    2583 scope.go:117] "RemoveContainer" containerID="c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"
Jul 28 18:03:46.221521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.221506    2583 scope.go:117] "RemoveContainer" containerID="4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"
Jul 28 18:03:46.228119 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.228097    2583 scope.go:117] "RemoveContainer" containerID="58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"
Jul 28 18:03:46.233564 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.233542    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:03:46.234932 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.234894    2583 scope.go:117] "RemoveContainer" containerID="21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"
Jul 28 18:03:46.238559 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.238534    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:03:46.241997 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.241974    2583 scope.go:117] "RemoveContainer" containerID="c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"
Jul 28 18:03:46.249011 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.248995    2583 scope.go:117] "RemoveContainer" containerID="ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"
Jul 28 18:03:46.255634 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.255616    2583 scope.go:117] "RemoveContainer" containerID="993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"
Jul 28 18:03:46.255928 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:46.255896    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": container with ID starting with 993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6 not found: ID does not exist" containerID="993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"
Jul 28 18:03:46.255980 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.255930    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"} err="failed to get container status \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": rpc error: code = NotFound desc = could not find container \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": container with ID starting with 993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6 not found: ID does not exist"
Jul 28 18:03:46.255980 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.255948    2583 scope.go:117] "RemoveContainer" containerID="c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"
Jul 28 18:03:46.256206 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:46.256189    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": container with ID starting with c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40 not found: ID does not exist" containerID="c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"
Jul 28 18:03:46.256248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.256213    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"} err="failed to get container status \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": rpc error: code = NotFound desc = could not find container \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": container with ID starting with c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40 not found: ID does not exist"
Jul 28 18:03:46.256248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.256228    2583 scope.go:117] "RemoveContainer" containerID="4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"
Jul 28 18:03:46.256481 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:46.256457    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": container with ID starting with 4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c not found: ID does not exist" containerID="4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"
Jul 28 18:03:46.256530 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.256477    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"} err="failed to get container status \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": rpc error: code = NotFound desc = could not find container \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": container with ID starting with 4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c not found: ID does not exist"
Jul 28 18:03:46.256530 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.256490    2583 scope.go:117] "RemoveContainer" containerID="58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"
Jul 28 18:03:46.256735 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:46.256717    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": container with ID starting with 58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2 not found: ID does not exist" containerID="58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"
Jul 28 18:03:46.256773 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.256748    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"} err="failed to get container status \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": rpc error: code = NotFound desc = could not find container \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": container with ID starting with 58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2 not found: ID does not exist"
Jul 28 18:03:46.256773 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.256763    2583 scope.go:117] "RemoveContainer" containerID="21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"
Jul 28 18:03:46.257015 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:46.256995    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": container with ID starting with 21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22 not found: ID does not exist" containerID="21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"
Jul 28 18:03:46.257072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.257019    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"} err="failed to get container status \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": rpc error: code = NotFound desc = could not find container \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": container with ID starting with 21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22 not found: ID does not exist"
Jul 28 18:03:46.257072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.257032    2583 scope.go:117] "RemoveContainer" containerID="c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"
Jul 28 18:03:46.257279 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:46.257262    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": container with ID starting with c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f not found: ID does not exist" containerID="c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"
Jul 28 18:03:46.257350 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.257286    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"} err="failed to get container status \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": rpc error: code = NotFound desc = could not find container \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": container with ID starting with c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f not found: ID does not exist"
Jul 28 18:03:46.257350 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.257307    2583 scope.go:117] "RemoveContainer" containerID="ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"
Jul 28 18:03:46.257554 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:03:46.257536    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": container with ID starting with ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c not found: ID does not exist" containerID="ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"
Jul 28 18:03:46.257610 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.257559    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"} err="failed to get container status \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": rpc error: code = NotFound desc = could not find container \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": container with ID starting with ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c not found: ID does not exist"
Jul 28 18:03:46.257610 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.257578    2583 scope.go:117] "RemoveContainer" containerID="993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"
Jul 28 18:03:46.257793 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.257776    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"} err="failed to get container status \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": rpc error: code = NotFound desc = could not find container \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": container with ID starting with 993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6 not found: ID does not exist"
Jul 28 18:03:46.257855 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.257793    2583 scope.go:117] "RemoveContainer" containerID="c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"
Jul 28 18:03:46.258066 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.258040    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"} err="failed to get container status \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": rpc error: code = NotFound desc = could not find container \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": container with ID starting with c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40 not found: ID does not exist"
Jul 28 18:03:46.258109 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.258068    2583 scope.go:117] "RemoveContainer" containerID="4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"
Jul 28 18:03:46.258327 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.258310    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"} err="failed to get container status \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": rpc error: code = NotFound desc = could not find container \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": container with ID starting with 4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c not found: ID does not exist"
Jul 28 18:03:46.258376 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.258330    2583 scope.go:117] "RemoveContainer" containerID="58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"
Jul 28 18:03:46.258562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.258543    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"} err="failed to get container status \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": rpc error: code = NotFound desc = could not find container \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": container with ID starting with 58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2 not found: ID does not exist"
Jul 28 18:03:46.258609 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.258562    2583 scope.go:117] "RemoveContainer" containerID="21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"
Jul 28 18:03:46.258793 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.258775    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"} err="failed to get container status \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": rpc error: code = NotFound desc = could not find container \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": container with ID starting with 21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22 not found: ID does not exist"
Jul 28 18:03:46.258862 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.258794    2583 scope.go:117] "RemoveContainer" containerID="c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"
Jul 28 18:03:46.259019 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.258999    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"} err="failed to get container status \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": rpc error: code = NotFound desc = could not find container \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": container with ID starting with c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f not found: ID does not exist"
Jul 28 18:03:46.259063 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.259020    2583 scope.go:117] "RemoveContainer" containerID="ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"
Jul 28 18:03:46.259213 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.259196    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"} err="failed to get container status \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": rpc error: code = NotFound desc = could not find container \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": container with ID starting with ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c not found: ID does not exist"
Jul 28 18:03:46.259259 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.259214    2583 scope.go:117] "RemoveContainer" containerID="993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"
Jul 28 18:03:46.259446 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.259428    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"} err="failed to get container status \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": rpc error: code = NotFound desc = could not find container \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": container with ID starting with 993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6 not found: ID does not exist"
Jul 28 18:03:46.259491 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.259447    2583 scope.go:117] "RemoveContainer" containerID="c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"
Jul 28 18:03:46.259670 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.259653    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"} err="failed to get container status \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": rpc error: code = NotFound desc = could not find container \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": container with ID starting with c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40 not found: ID does not exist"
Jul 28 18:03:46.259737 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.259672    2583 scope.go:117] "RemoveContainer" containerID="4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"
Jul 28 18:03:46.259932 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.259916    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"} err="failed to get container status \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": rpc error: code = NotFound desc = could not find container \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": container with ID starting with 4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c not found: ID does not exist"
Jul 28 18:03:46.259979 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.259933    2583 scope.go:117] "RemoveContainer" containerID="58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"
Jul 28 18:03:46.260139 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.260113    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"} err="failed to get container status \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": rpc error: code = NotFound desc = could not find container \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": container with ID starting with 58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2 not found: ID does not exist"
Jul 28 18:03:46.260200 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.260138    2583 scope.go:117] "RemoveContainer" containerID="21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"
Jul 28 18:03:46.260364 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.260347    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"} err="failed to get container status \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": rpc error: code = NotFound desc = could not find container \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": container with ID starting with 21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22 not found: ID does not exist"
Jul 28 18:03:46.260407 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.260365    2583 scope.go:117] "RemoveContainer" containerID="c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"
Jul 28 18:03:46.260577 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.260562    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"} err="failed to get container status \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": rpc error: code = NotFound desc = could not find container \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": container with ID starting with c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f not found: ID does not exist"
Jul 28 18:03:46.260621 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.260577    2583 scope.go:117] "RemoveContainer" containerID="ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"
Jul 28 18:03:46.260808 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.260790    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"} err="failed to get container status \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": rpc error: code = NotFound desc = could not find container \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": container with ID starting with ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c not found: ID does not exist"
Jul 28 18:03:46.260869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.260808    2583 scope.go:117] "RemoveContainer" containerID="993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"
Jul 28 18:03:46.261048 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.261031    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"} err="failed to get container status \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": rpc error: code = NotFound desc = could not find container \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": container with ID starting with 993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6 not found: ID does not exist"
Jul 28 18:03:46.261099 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.261048    2583 scope.go:117] "RemoveContainer" containerID="c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"
Jul 28 18:03:46.261254 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.261238    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"} err="failed to get container status \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": rpc error: code = NotFound desc = could not find container \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": container with ID starting with c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40 not found: ID does not exist"
Jul 28 18:03:46.261296 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.261255    2583 scope.go:117] "RemoveContainer" containerID="4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"
Jul 28 18:03:46.261467 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.261449    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"} err="failed to get container status \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": rpc error: code = NotFound desc = could not find container \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": container with ID starting with 4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c not found: ID does not exist"
Jul 28 18:03:46.261519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.261468    2583 scope.go:117] "RemoveContainer" containerID="58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"
Jul 28 18:03:46.261689 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.261673    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"} err="failed to get container status \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": rpc error: code = NotFound desc = could not find container \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": container with ID starting with 58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2 not found: ID does not exist"
Jul 28 18:03:46.261731 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.261689    2583 scope.go:117] "RemoveContainer" containerID="21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"
Jul 28 18:03:46.262004 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.261983    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"} err="failed to get container status \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": rpc error: code = NotFound desc = could not find container \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": container with ID starting with 21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22 not found: ID does not exist"
Jul 28 18:03:46.262065 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.262005    2583 scope.go:117] "RemoveContainer" containerID="c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"
Jul 28 18:03:46.262244 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.262218    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"} err="failed to get container status \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": rpc error: code = NotFound desc = could not find container \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": container with ID starting with c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f not found: ID does not exist"
Jul 28 18:03:46.262244 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.262244    2583 scope.go:117] "RemoveContainer" containerID="ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"
Jul 28 18:03:46.262632 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.262601    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"} err="failed to get container status \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": rpc error: code = NotFound desc = could not find container \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": container with ID starting with ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c not found: ID does not exist"
Jul 28 18:03:46.262737 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.262632    2583 scope.go:117] "RemoveContainer" containerID="993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"
Jul 28 18:03:46.263042 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.263017    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"} err="failed to get container status \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": rpc error: code = NotFound desc = could not find container \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": container with ID starting with 993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6 not found: ID does not exist"
Jul 28 18:03:46.263126 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.263050    2583 scope.go:117] "RemoveContainer" containerID="c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"
Jul 28 18:03:46.263284 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.263257    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"} err="failed to get container status \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": rpc error: code = NotFound desc = could not find container \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": container with ID starting with c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40 not found: ID does not exist"
Jul 28 18:03:46.263284 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.263280    2583 scope.go:117] "RemoveContainer" containerID="4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"
Jul 28 18:03:46.263490 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.263459    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"} err="failed to get container status \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": rpc error: code = NotFound desc = could not find container \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": container with ID starting with 4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c not found: ID does not exist"
Jul 28 18:03:46.263490 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.263481    2583 scope.go:117] "RemoveContainer" containerID="58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"
Jul 28 18:03:46.263711 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.263693    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"} err="failed to get container status \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": rpc error: code = NotFound desc = could not find container \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": container with ID starting with 58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2 not found: ID does not exist"
Jul 28 18:03:46.263751 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.263712    2583 scope.go:117] "RemoveContainer" containerID="21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"
Jul 28 18:03:46.264008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.263991    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"} err="failed to get container status \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": rpc error: code = NotFound desc = could not find container \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": container with ID starting with 21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22 not found: ID does not exist"
Jul 28 18:03:46.264058 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264008    2583 scope.go:117] "RemoveContainer" containerID="c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"
Jul 28 18:03:46.264220 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264204    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"} err="failed to get container status \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": rpc error: code = NotFound desc = could not find container \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": container with ID starting with c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f not found: ID does not exist"
Jul 28 18:03:46.264264 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264221    2583 scope.go:117] "RemoveContainer" containerID="ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"
Jul 28 18:03:46.264458 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264441    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"} err="failed to get container status \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": rpc error: code = NotFound desc = could not find container \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": container with ID starting with ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c not found: ID does not exist"
Jul 28 18:03:46.264506 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264458    2583 scope.go:117] "RemoveContainer" containerID="993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"
Jul 28 18:03:46.264623 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264604    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:03:46.264671 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264657    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6"} err="failed to get container status \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": rpc error: code = NotFound desc = could not find container \"993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6\": container with ID starting with 993578ac6611695ce65c687254b5eadec2f5a714e84df8938b6fa7a491c12bc6 not found: ID does not exist"
Jul 28 18:03:46.264709 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264673    2583 scope.go:117] "RemoveContainer" containerID="c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"
Jul 28 18:03:46.264932 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264914    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40"} err="failed to get container status \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": rpc error: code = NotFound desc = could not find container \"c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40\": container with ID starting with c4647247674064ec6b2932ab8f674d4c55105dc176727497d432b97d2b07ee40 not found: ID does not exist"
Jul 28 18:03:46.265009 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264934    2583 scope.go:117] "RemoveContainer" containerID="4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"
Jul 28 18:03:46.265009 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264968    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy-thanos"
Jul 28 18:03:46.265009 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264981    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy-thanos"
Jul 28 18:03:46.265009 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264989    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="init-config-reloader"
Jul 28 18:03:46.265009 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.264995    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="init-config-reloader"
Jul 28 18:03:46.265009 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265003    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="thanos-sidecar"
Jul 28 18:03:46.265009 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265008    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="thanos-sidecar"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265015    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265020    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265032    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="prometheus"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265037    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="prometheus"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265043    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="config-reloader"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265050    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="config-reloader"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265060    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy-web"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265066    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy-web"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265133    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy-web"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265145    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="config-reloader"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265153    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="thanos-sidecar"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265166    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265174    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="prometheus"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265170    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c"} err="failed to get container status \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": rpc error: code = NotFound desc = could not find container \"4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c\": container with ID starting with 4ca84b0002632d07596407a833c0d4243c7f7df2a91770ec49db73e6a60cb81c not found: ID does not exist"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265191    2583 scope.go:117] "RemoveContainer" containerID="58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265183    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" containerName="kube-rbac-proxy-thanos"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265476    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2"} err="failed to get container status \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": rpc error: code = NotFound desc = could not find container \"58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2\": container with ID starting with 58a2b028650ff4d4c244625a8583a0b5c7f41a9a0a651ff70e55fcd6e5660ca2 not found: ID does not exist"
Jul 28 18:03:46.265504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265496    2583 scope.go:117] "RemoveContainer" containerID="21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"
Jul 28 18:03:46.266357 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265741    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22"} err="failed to get container status \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": rpc error: code = NotFound desc = could not find container \"21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22\": container with ID starting with 21a6d62a0acb5177eb47e7ddde7797f1799d7a87d0af35db5b887059df50bd22 not found: ID does not exist"
Jul 28 18:03:46.266357 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.265763    2583 scope.go:117] "RemoveContainer" containerID="c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"
Jul 28 18:03:46.266357 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.266026    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f"} err="failed to get container status \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": rpc error: code = NotFound desc = could not find container \"c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f\": container with ID starting with c8aa9277fca2996d17c4df8c6b8055a7299aeaf976b577b60bbb2d1d0f61481f not found: ID does not exist"
Jul 28 18:03:46.266357 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.266047    2583 scope.go:117] "RemoveContainer" containerID="ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"
Jul 28 18:03:46.266357 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.266297    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c"} err="failed to get container status \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": rpc error: code = NotFound desc = could not find container \"ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c\": container with ID starting with ee582d3e95c47a859ebbafc14d8ddaa8fcedf87172d5f29044d968255a13ba8c not found: ID does not exist"
Jul 28 18:03:46.271443 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.271425    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.274492 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.274471    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-d5dll\""
Jul 28 18:03:46.274639 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.274600    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 18:03:46.274754 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.274670    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 18:03:46.274754 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.274676    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3ur2gh26chm9q\""
Jul 28 18:03:46.274754 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.274741    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 18:03:46.274988 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.274974    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 18:03:46.275032 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.275022    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 18:03:46.275400 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.275375    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 18:03:46.275400 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.275382    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 18:03:46.275735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.275718    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 18:03:46.275800 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.275747    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 18:03:46.275800 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.275752    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 18:03:46.275931 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.275723    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 18:03:46.276151 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.276132    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 18:03:46.279133 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.279113    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 18:03:46.281161 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.281099    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 18:03:46.283484 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.283464    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:03:46.389339 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389298    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.389339 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389342    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.389592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389375    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.389592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389396    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.389592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389418    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.389592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389455    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.389592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389478    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.389592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.389592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389524    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.389592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.390008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.390008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389711    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.390008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389743    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.390008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-config\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.390008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389847    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.390008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389900    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.390008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389932    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.390008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.390008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.389993    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.390306 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.390044    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwbpk\" (UniqueName: \"kubernetes.io/projected/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-kube-api-access-wwbpk\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491114    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491166    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491212 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491196    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491229    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwbpk\" (UniqueName: \"kubernetes.io/projected/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-kube-api-access-wwbpk\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491270    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491296    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491330    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491354    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491377    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491419    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491447    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491946 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491946 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491496    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491946 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491524    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491946 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491547    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491946 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491946 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491602    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491946 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491627    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-config\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491946 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491663    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.491946 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.491722    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.492380 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.492249    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.492788 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.492528    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.492933 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.492915    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.494565 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.494269    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.494565 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.494497    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.494565 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.494501    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.494565 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.494501    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.494884 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.494574    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.494884 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.494647    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.495548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.495125    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.495548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.495317    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.495548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.495375    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.495548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.495459    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.495548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.495475    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.495796 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.495627    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.496740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.496715    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.497039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.497012    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-config\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.497515 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.497494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.497563 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.497494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.500103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.500084    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwbpk\" (UniqueName: \"kubernetes.io/projected/a36fd72a-e1bb-4048-8be3-ddb7244c0cbf-kube-api-access-wwbpk\") pod \"prometheus-k8s-0\" (UID: \"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.581265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.581223    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:03:46.720660 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:46.720628    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:03:46.723838 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:03:46.723799    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36fd72a_e1bb_4048_8be3_ddb7244c0cbf.slice/crio-0ea165729113e76b89b8edd9f006023235d0642a5ee071eaf13d7a9ad1eee8bd WatchSource:0}: Error finding container 0ea165729113e76b89b8edd9f006023235d0642a5ee071eaf13d7a9ad1eee8bd: Status 404 returned error can't find the container with id 0ea165729113e76b89b8edd9f006023235d0642a5ee071eaf13d7a9ad1eee8bd
Jul 28 18:03:47.216288 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:47.216249    2583 generic.go:356] "Generic (PLEG): container finished" podID="a36fd72a-e1bb-4048-8be3-ddb7244c0cbf" containerID="84fd660992154b155b71bc9b86f6ccddae69b2ac57cec162f4b08b92d31e3a92" exitCode=0
Jul 28 18:03:47.216659 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:47.216326    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf","Type":"ContainerDied","Data":"84fd660992154b155b71bc9b86f6ccddae69b2ac57cec162f4b08b92d31e3a92"}
Jul 28 18:03:47.216659 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:47.216413    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf","Type":"ContainerStarted","Data":"0ea165729113e76b89b8edd9f006023235d0642a5ee071eaf13d7a9ad1eee8bd"}
Jul 28 18:03:47.411376 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:47.410582    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bcb9662-1721-44c5-a841-0156bff00d32" path="/var/lib/kubelet/pods/0bcb9662-1721-44c5-a841-0156bff00d32/volumes"
Jul 28 18:03:48.224969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:48.224930    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf","Type":"ContainerStarted","Data":"11220f4e4199bb42f1d6019ee052c288a44ed6ecf74dcc87582ba573b97c1e34"}
Jul 28 18:03:48.225393 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:48.225015    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf","Type":"ContainerStarted","Data":"df7870b1ace837a965cdaa5955aeda4f5b6d36f9195b764995dfae05fa088b98"}
Jul 28 18:03:48.225393 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:48.225055    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf","Type":"ContainerStarted","Data":"146b417cc458a37a40661791b6aaf16c5d4cc80d45a7950ea4ff7766a8a234d4"}
Jul 28 18:03:48.225393 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:48.225097    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf","Type":"ContainerStarted","Data":"0d25ce966d3760a7b4f35f7c2557784d9ab82fde7a42ba32bc9892e680c4e1be"}
Jul 28 18:03:48.225393 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:48.225133    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf","Type":"ContainerStarted","Data":"abd180e6e530221df7df0cceb6988b5f4112ad777a476d25db0b16b5b32cf857"}
Jul 28 18:03:48.225393 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:48.225167    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a36fd72a-e1bb-4048-8be3-ddb7244c0cbf","Type":"ContainerStarted","Data":"547b48765980455348016e7881368de25a2e47c5bb468542ab91d812447e7233"}
Jul 28 18:03:48.258699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:48.258644    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.241459986 podStartE2EDuration="2.258623826s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.16384ms" isStatefulPod=false podCreationTimestamp="2026-07-28 18:03:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:03:48.257910327 +0000 UTC m=+179.334472324" watchObservedRunningTime="2026-07-28 18:03:48.258623826 +0000 UTC m=+179.335185825"
Jul 28 18:03:50.825166 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:50.825128    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:50.883888 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:50.883856    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:51.582255 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:03:51.582216    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:04:02.092910 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:02.092872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:09.722070 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:09.722027    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:09.789211 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:09.789179    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:11.370272 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:11.370229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:28.882706 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:28.882623    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:37.894649 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:37.894608    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:46.581853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:46.581800    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:04:46.596874 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:46.596845    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:04:47.461630 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:47.461604    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:04:55.501302 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:55.501265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:55.579148 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:04:55.579119    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:04.329659 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:04.329603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:04.403874 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:04.403842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:07.475877 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:07.475850    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:07.578384 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:07.578350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:14.979525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:14.979435    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:15.079536 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:15.079501    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:15.400309 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:15.400209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:15.511884 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:15.511845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:15.598982 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:15.598942    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:49.291691 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:49.291663    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:05:49.294561 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:49.294536    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:05:49.302642 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:49.302623    2583 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 18:05:51.584757 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:51.584713    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:51.660965 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:05:51.660918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:13.672122 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:13.672087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:15.724232 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:15.724189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:17.997879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:17.997809    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:18.082864 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:18.082745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:19.243975 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:19.243944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:19.343164 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:19.343126    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:19.900905 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:19.900875    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:20.026879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:20.026848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:21.479490 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:21.479453    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:21.895662 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:21.895573    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:24.488929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:24.488896    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:24.546704 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:24.546664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:29.291305 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:29.291273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:43.271378 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:43.271299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:43.370783 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:43.370743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:45.329132 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.329097    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"]
Jul 28 18:06:45.332028 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.332010    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"
Jul 28 18:06:45.335113 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.335095    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5j2nc\""
Jul 28 18:06:45.335336 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.335323    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 18:06:45.336578 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.336564    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 18:06:45.345865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.345839    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"]
Jul 28 18:06:45.405930 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.405899    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g6wj\" (UniqueName: \"kubernetes.io/projected/5fd2a3ed-6a11-4158-902f-e0ff128027c4-kube-api-access-8g6wj\") pod \"openshift-lws-operator-fd8ccff4c-d27x7\" (UID: \"5fd2a3ed-6a11-4158-902f-e0ff128027c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"
Jul 28 18:06:45.406088 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.405972    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fd2a3ed-6a11-4158-902f-e0ff128027c4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d27x7\" (UID: \"5fd2a3ed-6a11-4158-902f-e0ff128027c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"
Jul 28 18:06:45.506970 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.506934    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fd2a3ed-6a11-4158-902f-e0ff128027c4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d27x7\" (UID: \"5fd2a3ed-6a11-4158-902f-e0ff128027c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"
Jul 28 18:06:45.507107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.506990    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g6wj\" (UniqueName: \"kubernetes.io/projected/5fd2a3ed-6a11-4158-902f-e0ff128027c4-kube-api-access-8g6wj\") pod \"openshift-lws-operator-fd8ccff4c-d27x7\" (UID: \"5fd2a3ed-6a11-4158-902f-e0ff128027c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"
Jul 28 18:06:45.507313 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.507283    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fd2a3ed-6a11-4158-902f-e0ff128027c4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d27x7\" (UID: \"5fd2a3ed-6a11-4158-902f-e0ff128027c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"
Jul 28 18:06:45.516839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.516784    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g6wj\" (UniqueName: \"kubernetes.io/projected/5fd2a3ed-6a11-4158-902f-e0ff128027c4-kube-api-access-8g6wj\") pod \"openshift-lws-operator-fd8ccff4c-d27x7\" (UID: \"5fd2a3ed-6a11-4158-902f-e0ff128027c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"
Jul 28 18:06:45.651854 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.651753    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"
Jul 28 18:06:45.783728 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.783704    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7"]
Jul 28 18:06:45.786584 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:06:45.786561    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fd2a3ed_6a11_4158_902f_e0ff128027c4.slice/crio-a2dff8dda23fbd923da352cf5a8d79d71af97f6b34f89f41060b9dc958e85e80 WatchSource:0}: Error finding container a2dff8dda23fbd923da352cf5a8d79d71af97f6b34f89f41060b9dc958e85e80: Status 404 returned error can't find the container with id a2dff8dda23fbd923da352cf5a8d79d71af97f6b34f89f41060b9dc958e85e80
Jul 28 18:06:45.788461 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.788446    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:06:45.900477 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:45.900436    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7" event={"ID":"5fd2a3ed-6a11-4158-902f-e0ff128027c4","Type":"ContainerStarted","Data":"a2dff8dda23fbd923da352cf5a8d79d71af97f6b34f89f41060b9dc958e85e80"}
Jul 28 18:06:46.825159 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:46.825121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:46.894045 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:46.894009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:48.916051 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:48.916018    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7" event={"ID":"5fd2a3ed-6a11-4158-902f-e0ff128027c4","Type":"ContainerStarted","Data":"61805ac99d20d1db47bafff296cdcb6c08159df659ed1f61b733f7d88fef22fa"}
Jul 28 18:06:48.942480 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:48.942429    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d27x7" podStartSLOduration=1.632516372 podStartE2EDuration="3.9424146s" totalImagesPullingTime="2.309898228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:06:48.938319067 +0000 UTC m=+360.014881064" watchObservedRunningTime="2026-07-28 18:06:48.9424146 +0000 UTC m=+360.018976600"
Jul 28 18:06:49.617407 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:49.617364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:49.756741 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:49.756706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:51.794115 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:06:51.794078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:02.015322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.015267    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:02.075676 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.075640    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"]
Jul 28 18:07:02.079273 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.079251    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.083966 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.083944    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 18:07:02.084232 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.084216    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-d6fnj\""
Jul 28 18:07:02.084461 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.084446    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 18:07:02.084541 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.084483    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 18:07:02.102022 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.101991    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"]
Jul 28 18:07:02.107413 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.107378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:02.150180 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.150145    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a8ab0c02-5d57-4e79-863d-2000d0119e5a-metrics-cert\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.150350 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.150199    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8ab0c02-5d57-4e79-863d-2000d0119e5a-cert\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.150350 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.150229    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmmpb\" (UniqueName: \"kubernetes.io/projected/a8ab0c02-5d57-4e79-863d-2000d0119e5a-kube-api-access-mmmpb\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.150350 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.150252    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a8ab0c02-5d57-4e79-863d-2000d0119e5a-manager-config\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.251663 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.251613    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a8ab0c02-5d57-4e79-863d-2000d0119e5a-manager-config\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.251869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.251713    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a8ab0c02-5d57-4e79-863d-2000d0119e5a-metrics-cert\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.251869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.251759    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8ab0c02-5d57-4e79-863d-2000d0119e5a-cert\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.251869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.251781    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmmpb\" (UniqueName: \"kubernetes.io/projected/a8ab0c02-5d57-4e79-863d-2000d0119e5a-kube-api-access-mmmpb\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.252327 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.252304    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a8ab0c02-5d57-4e79-863d-2000d0119e5a-manager-config\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.254243 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.254222    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a8ab0c02-5d57-4e79-863d-2000d0119e5a-metrics-cert\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.254332 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.254260    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8ab0c02-5d57-4e79-863d-2000d0119e5a-cert\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.273204 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.273131    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmmpb\" (UniqueName: \"kubernetes.io/projected/a8ab0c02-5d57-4e79-863d-2000d0119e5a-kube-api-access-mmmpb\") pod \"lws-controller-manager-84df6f5b9b-vb59l\" (UID: \"a8ab0c02-5d57-4e79-863d-2000d0119e5a\") " pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.387802 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.387761    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:02.528621 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.528595    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"]
Jul 28 18:07:02.530964 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:07:02.530941    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ab0c02_5d57_4e79_863d_2000d0119e5a.slice/crio-82a29d02335eba4fcdf15968c925f5e4680fa894e06e79bc649ea38011fc8069 WatchSource:0}: Error finding container 82a29d02335eba4fcdf15968c925f5e4680fa894e06e79bc649ea38011fc8069: Status 404 returned error can't find the container with id 82a29d02335eba4fcdf15968c925f5e4680fa894e06e79bc649ea38011fc8069
Jul 28 18:07:02.970355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:02.970322    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l" event={"ID":"a8ab0c02-5d57-4e79-863d-2000d0119e5a","Type":"ContainerStarted","Data":"82a29d02335eba4fcdf15968c925f5e4680fa894e06e79bc649ea38011fc8069"}
Jul 28 18:07:04.982115 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:04.982076    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l" event={"ID":"a8ab0c02-5d57-4e79-863d-2000d0119e5a","Type":"ContainerStarted","Data":"8380b50488bada0e5226f111aa8f35b3b3b4be4ffb9a3270b686ace4389502f1"}
Jul 28 18:07:04.982533 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:04.982200    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:05.002043 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:05.001991    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l" podStartSLOduration=1.3689544900000001 podStartE2EDuration="3.001976305s" totalImagesPullingTime="1.633021815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:07:05.001293643 +0000 UTC m=+376.077855640" watchObservedRunningTime="2026-07-28 18:07:05.001976305 +0000 UTC m=+376.078538301"
Jul 28 18:07:07.369859 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:07.369805    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:07.467332 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:07.467301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:07.997769 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:07.997733    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:08.214729 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.214691    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:08.318409 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.318322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:08.354828 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.354779    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"]
Jul 28 18:07:08.358356 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.358335    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.361657 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.361639    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 18:07:08.361754 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.361666    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 18:07:08.361847 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.361794    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sz7wn\""
Jul 28 18:07:08.361961 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.361946    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 18:07:08.362270 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.362256    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 18:07:08.393249 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.393215    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"]
Jul 28 18:07:08.510260 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.510227    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb8a4236-d236-4a19-aced-a4778d0512e1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.510482 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.510273    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77jp7\" (UniqueName: \"kubernetes.io/projected/eb8a4236-d236-4a19-aced-a4778d0512e1-kube-api-access-77jp7\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.510482 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.510349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb8a4236-d236-4a19-aced-a4778d0512e1-tmp\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.510482 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.510397    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb8a4236-d236-4a19-aced-a4778d0512e1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.510482 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.510441    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb8a4236-d236-4a19-aced-a4778d0512e1-manifests\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.520099 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.520070    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:08.611396 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.611313    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb8a4236-d236-4a19-aced-a4778d0512e1-tmp\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.611396 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.611347    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb8a4236-d236-4a19-aced-a4778d0512e1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.611396 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.611370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb8a4236-d236-4a19-aced-a4778d0512e1-manifests\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.611661 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.611462    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb8a4236-d236-4a19-aced-a4778d0512e1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.611661 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.611490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77jp7\" (UniqueName: \"kubernetes.io/projected/eb8a4236-d236-4a19-aced-a4778d0512e1-kube-api-access-77jp7\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.611863 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.611795    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb8a4236-d236-4a19-aced-a4778d0512e1-manifests\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.613667 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.613641    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb8a4236-d236-4a19-aced-a4778d0512e1-tmp\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.613785 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.613768    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb8a4236-d236-4a19-aced-a4778d0512e1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.613938 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.613921    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb8a4236-d236-4a19-aced-a4778d0512e1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.626739 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.626715    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77jp7\" (UniqueName: \"kubernetes.io/projected/eb8a4236-d236-4a19-aced-a4778d0512e1-kube-api-access-77jp7\") pod \"opendatahub-operator-controller-manager-7795fd7c74-rxz9c\" (UID: \"eb8a4236-d236-4a19-aced-a4778d0512e1\") " pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.627530 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.627505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:08.668687 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.668662    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:08.728196 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.728153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:08.813734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.813690    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"]
Jul 28 18:07:08.817896 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:07:08.817867    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8a4236_d236_4a19_aced_a4778d0512e1.slice/crio-ad6399c850e90be04935c17706f21cfbc7d3f8a642600c6a40671607cd83b20c WatchSource:0}: Error finding container ad6399c850e90be04935c17706f21cfbc7d3f8a642600c6a40671607cd83b20c: Status 404 returned error can't find the container with id ad6399c850e90be04935c17706f21cfbc7d3f8a642600c6a40671607cd83b20c
Jul 28 18:07:08.818661 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.818631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:08.886126 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.886056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:08.990455 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:08.990425    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:09.000606 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:09.000583    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c" event={"ID":"eb8a4236-d236-4a19-aced-a4778d0512e1","Type":"ContainerStarted","Data":"ad6399c850e90be04935c17706f21cfbc7d3f8a642600c6a40671607cd83b20c"}
Jul 28 18:07:09.087235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:09.087180    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:12.017194 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:12.017162    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c" event={"ID":"eb8a4236-d236-4a19-aced-a4778d0512e1","Type":"ContainerStarted","Data":"7d7c93c399255ea91eeb4e746cddc2680f14125377eaa57dad514c0882138054"}
Jul 28 18:07:13.023229 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:13.023196    2583 generic.go:356] "Generic (PLEG): container finished" podID="eb8a4236-d236-4a19-aced-a4778d0512e1" containerID="7d7c93c399255ea91eeb4e746cddc2680f14125377eaa57dad514c0882138054" exitCode=0
Jul 28 18:07:13.023552 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:13.023289    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c" event={"ID":"eb8a4236-d236-4a19-aced-a4778d0512e1","Type":"ContainerDied","Data":"7d7c93c399255ea91eeb4e746cddc2680f14125377eaa57dad514c0882138054"}
Jul 28 18:07:14.030739 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:14.030699    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c" event={"ID":"eb8a4236-d236-4a19-aced-a4778d0512e1","Type":"ContainerStarted","Data":"78620fdb822960d25a349d74dcfe788b39dd914b5367e1b1dd997bbaf640b6fa"}
Jul 28 18:07:14.031204 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:14.030878    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:14.059485 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:14.059428    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c" podStartSLOduration=1.5744746040000002 podStartE2EDuration="6.059414588s" totalImagesPullingTime="3.441862354s" totalInitContainerRuntime="1.04307763s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:07:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:07:14.058092159 +0000 UTC m=+385.134654155" watchObservedRunningTime="2026-07-28 18:07:14.059414588 +0000 UTC m=+385.135976586"
Jul 28 18:07:15.988594 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:15.988559    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84df6f5b9b-vb59l"
Jul 28 18:07:19.284396 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:19.284361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:19.377942 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:19.377902    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:19.723619 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:19.723585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:20.232336 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:20.232277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:20.325256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:20.325210    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:25.037677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:25.037647    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7795fd7c74-rxz9c"
Jul 28 18:07:27.573387 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:27.573353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:27.671693 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:27.671656    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:28.436446 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:28.436409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:28.540588 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:28.540553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:28.639337 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:28.639302    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:29.120665 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:29.120631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:29.227279 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:29.227247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:37.855731 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:37.855686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:37.946973 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:37.946930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:38.292348 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:38.292312    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:38.458667 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:38.458628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:38.685561 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:38.685486    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:39.173025 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:39.172989    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:39.278757 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:39.278719    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:39.378302 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:39.378263    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:39.549305 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:39.549271    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:40.503198 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:40.503163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:40.603473 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:40.603437    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:41.987353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:41.987319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:42.180405 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:42.180369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:42.241686 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:42.241593    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:45.541359 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.541322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:45.648131 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.648101    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:45.890425 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.890343    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"]
Jul 28 18:07:45.896967 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.896943    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:45.900205 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.900176    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 18:07:45.900205 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.900190    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-s8pwv\""
Jul 28 18:07:45.910148 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.910118    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"]
Jul 28 18:07:45.916560 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.916529    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:45.930306 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.930269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptwkz\" (UniqueName: \"kubernetes.io/projected/039a55ec-d600-4f19-b3ad-76642f0b42b2-kube-api-access-ptwkz\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:45.930583 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.930557    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:45.930689 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.930603    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:45.930689 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.930651    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:45.930775 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.930742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:45.930845 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.930780    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:45.930845 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.930800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:45.930957 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.930874    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/039a55ec-d600-4f19-b3ad-76642f0b42b2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:45.931025 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:45.931000    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.031705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.031671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.031705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.031706    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.031996 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.031748    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/039a55ec-d600-4f19-b3ad-76642f0b42b2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.031996 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.031783    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.031996 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.031842    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptwkz\" (UniqueName: \"kubernetes.io/projected/039a55ec-d600-4f19-b3ad-76642f0b42b2-kube-api-access-ptwkz\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.031996 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.031902    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.031996 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.031925    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.031996 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.031968    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.032291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.032008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.032340 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.032317    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.032437 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.032378    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.032437 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.032408    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.032698 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.032665    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.032887 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.032869    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/039a55ec-d600-4f19-b3ad-76642f0b42b2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.034105 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.034088    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.034333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.034312    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.040638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.040618    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/039a55ec-d600-4f19-b3ad-76642f0b42b2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.041979 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.041958    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptwkz\" (UniqueName: \"kubernetes.io/projected/039a55ec-d600-4f19-b3ad-76642f0b42b2-kube-api-access-ptwkz\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cqskvj\" (UID: \"039a55ec-d600-4f19-b3ad-76642f0b42b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.046513 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.046479    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:46.208558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.208513    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:46.291428 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.291386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:46.354087 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:46.354058    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"]
Jul 28 18:07:46.355681 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:07:46.355650    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039a55ec_d600_4f19_b3ad_76642f0b42b2.slice/crio-022468ecb042f6ace2550144f0df25f4391260203198f2fce47cce6ce236b68c WatchSource:0}: Error finding container 022468ecb042f6ace2550144f0df25f4391260203198f2fce47cce6ce236b68c: Status 404 returned error can't find the container with id 022468ecb042f6ace2550144f0df25f4391260203198f2fce47cce6ce236b68c
Jul 28 18:07:47.182347 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:47.182307    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj" event={"ID":"039a55ec-d600-4f19-b3ad-76642f0b42b2","Type":"ContainerStarted","Data":"022468ecb042f6ace2550144f0df25f4391260203198f2fce47cce6ce236b68c"}
Jul 28 18:07:49.164699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:49.164659    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:07:49.165091 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:49.164750    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:07:49.165091 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:49.164794    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:07:50.196895 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:50.196859    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj" event={"ID":"039a55ec-d600-4f19-b3ad-76642f0b42b2","Type":"ContainerStarted","Data":"041f6575c32c15ea84b591932b1b36eba4f13b1ea48c700e51ac40aed39931c5"}
Jul 28 18:07:50.209696 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:50.209669    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:50.214015 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:50.213991    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:50.221408 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:50.221363    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj" podStartSLOduration=2.414812225 podStartE2EDuration="5.221344502s" totalImagesPullingTime="2.806532277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:07:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:07:50.219103459 +0000 UTC m=+421.295665489" watchObservedRunningTime="2026-07-28 18:07:50.221344502 +0000 UTC m=+421.297906498"
Jul 28 18:07:51.201060 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:51.201033    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:07:51.202002 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:07:51.201984    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cqskvj"
Jul 28 18:08:29.140007 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.139957    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:29.236211 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.236165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:29.273036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.273006    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ffxk7"]
Jul 28 18:08:29.283085 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.283061    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ffxk7"
Jul 28 18:08:29.286402 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.286372    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 18:08:29.286973 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.286950    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ffxk7"]
Jul 28 18:08:29.287836 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.287697    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 18:08:29.287836 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.287714    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-h2x82\""
Jul 28 18:08:29.320010 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.319976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:29.326162 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.326141    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bmg5\" (UniqueName: \"kubernetes.io/projected/d01de4d9-e0fd-4544-8592-bd5c9d1c2233-kube-api-access-6bmg5\") pod \"kuadrant-operator-catalog-ffxk7\" (UID: \"d01de4d9-e0fd-4544-8592-bd5c9d1c2233\") " pod="kuadrant-system/kuadrant-operator-catalog-ffxk7"
Jul 28 18:08:29.427095 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.427023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bmg5\" (UniqueName: \"kubernetes.io/projected/d01de4d9-e0fd-4544-8592-bd5c9d1c2233-kube-api-access-6bmg5\") pod \"kuadrant-operator-catalog-ffxk7\" (UID: \"d01de4d9-e0fd-4544-8592-bd5c9d1c2233\") " pod="kuadrant-system/kuadrant-operator-catalog-ffxk7"
Jul 28 18:08:29.435872 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.435852    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bmg5\" (UniqueName: \"kubernetes.io/projected/d01de4d9-e0fd-4544-8592-bd5c9d1c2233-kube-api-access-6bmg5\") pod \"kuadrant-operator-catalog-ffxk7\" (UID: \"d01de4d9-e0fd-4544-8592-bd5c9d1c2233\") " pod="kuadrant-system/kuadrant-operator-catalog-ffxk7"
Jul 28 18:08:29.593987 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.593953    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ffxk7"
Jul 28 18:08:29.640292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.640233    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ffxk7"]
Jul 28 18:08:29.719391 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.719367    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ffxk7"]
Jul 28 18:08:29.722086 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:08:29.722058    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd01de4d9_e0fd_4544_8592_bd5c9d1c2233.slice/crio-d54020ddeff9bd61ee828a81ac2879ed3646493e98b3de80458ab98381f2b97e WatchSource:0}: Error finding container d54020ddeff9bd61ee828a81ac2879ed3646493e98b3de80458ab98381f2b97e: Status 404 returned error can't find the container with id d54020ddeff9bd61ee828a81ac2879ed3646493e98b3de80458ab98381f2b97e
Jul 28 18:08:29.853193 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.853167    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-878fw"]
Jul 28 18:08:29.857984 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.857969    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-878fw"
Jul 28 18:08:29.865630 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.865604    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-878fw"]
Jul 28 18:08:29.931592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:29.931561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvf8g\" (UniqueName: \"kubernetes.io/projected/6c4b138e-70e7-4fe6-bb25-c5e88cf74ac8-kube-api-access-hvf8g\") pod \"kuadrant-operator-catalog-878fw\" (UID: \"6c4b138e-70e7-4fe6-bb25-c5e88cf74ac8\") " pod="kuadrant-system/kuadrant-operator-catalog-878fw"
Jul 28 18:08:30.032949 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:30.032861    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvf8g\" (UniqueName: \"kubernetes.io/projected/6c4b138e-70e7-4fe6-bb25-c5e88cf74ac8-kube-api-access-hvf8g\") pod \"kuadrant-operator-catalog-878fw\" (UID: \"6c4b138e-70e7-4fe6-bb25-c5e88cf74ac8\") " pod="kuadrant-system/kuadrant-operator-catalog-878fw"
Jul 28 18:08:30.045094 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:30.045063    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvf8g\" (UniqueName: \"kubernetes.io/projected/6c4b138e-70e7-4fe6-bb25-c5e88cf74ac8-kube-api-access-hvf8g\") pod \"kuadrant-operator-catalog-878fw\" (UID: \"6c4b138e-70e7-4fe6-bb25-c5e88cf74ac8\") " pod="kuadrant-system/kuadrant-operator-catalog-878fw"
Jul 28 18:08:30.077712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:30.077679    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:30.167924 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:30.167893    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-878fw"
Jul 28 18:08:30.280489 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:30.280382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:30.306536 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:30.306515    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-878fw"]
Jul 28 18:08:30.308842 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:08:30.308800    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4b138e_70e7_4fe6_bb25_c5e88cf74ac8.slice/crio-20714dbe19754439d95af9453bb9a43ce4b5994a2421c44f5f6ef7aa330ecd90 WatchSource:0}: Error finding container 20714dbe19754439d95af9453bb9a43ce4b5994a2421c44f5f6ef7aa330ecd90: Status 404 returned error can't find the container with id 20714dbe19754439d95af9453bb9a43ce4b5994a2421c44f5f6ef7aa330ecd90
Jul 28 18:08:30.371067 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:30.371029    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-878fw" event={"ID":"6c4b138e-70e7-4fe6-bb25-c5e88cf74ac8","Type":"ContainerStarted","Data":"20714dbe19754439d95af9453bb9a43ce4b5994a2421c44f5f6ef7aa330ecd90"}
Jul 28 18:08:30.372317 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:30.372290    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ffxk7" event={"ID":"d01de4d9-e0fd-4544-8592-bd5c9d1c2233","Type":"ContainerStarted","Data":"d54020ddeff9bd61ee828a81ac2879ed3646493e98b3de80458ab98381f2b97e"}
Jul 28 18:08:32.384484 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:32.384444    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-878fw" event={"ID":"6c4b138e-70e7-4fe6-bb25-c5e88cf74ac8","Type":"ContainerStarted","Data":"5c511e8cac655efd6d02a2debbbd0018d04cd03cb9c8a8c67cb4d0b3aa053a73"}
Jul 28 18:08:32.385964 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:32.385941    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ffxk7" event={"ID":"d01de4d9-e0fd-4544-8592-bd5c9d1c2233","Type":"ContainerStarted","Data":"a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1"}
Jul 28 18:08:32.386075 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:32.385991    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ffxk7" podUID="d01de4d9-e0fd-4544-8592-bd5c9d1c2233" containerName="registry-server" containerID="cri-o://a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1" gracePeriod=2
Jul 28 18:08:32.402418 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:32.402374    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-878fw" podStartSLOduration=1.8093987729999998 podStartE2EDuration="3.402360461s" totalImagesPullingTime="1.592961688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:08:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:08:32.401434908 +0000 UTC m=+463.477996906" watchObservedRunningTime="2026-07-28 18:08:32.402360461 +0000 UTC m=+463.478922457"
Jul 28 18:08:32.419027 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:32.418979    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ffxk7" podStartSLOduration=1.241573038 podStartE2EDuration="3.418964184s" totalImagesPullingTime="2.177391146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:08:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:08:32.417752236 +0000 UTC m=+463.494314233" watchObservedRunningTime="2026-07-28 18:08:32.418964184 +0000 UTC m=+463.495526183"
Jul 28 18:08:32.620034 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:32.620010    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ffxk7"
Jul 28 18:08:32.659720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:32.659657    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bmg5\" (UniqueName: \"kubernetes.io/projected/d01de4d9-e0fd-4544-8592-bd5c9d1c2233-kube-api-access-6bmg5\") pod \"d01de4d9-e0fd-4544-8592-bd5c9d1c2233\" (UID: \"d01de4d9-e0fd-4544-8592-bd5c9d1c2233\") "
Jul 28 18:08:32.661654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:32.661626    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d01de4d9-e0fd-4544-8592-bd5c9d1c2233-kube-api-access-6bmg5" (OuterVolumeSpecName: "kube-api-access-6bmg5") pod "d01de4d9-e0fd-4544-8592-bd5c9d1c2233" (UID: "d01de4d9-e0fd-4544-8592-bd5c9d1c2233"). InnerVolumeSpecName "kube-api-access-6bmg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:08:32.760982 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:32.760952    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bmg5\" (UniqueName: \"kubernetes.io/projected/d01de4d9-e0fd-4544-8592-bd5c9d1c2233-kube-api-access-6bmg5\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:08:33.391603 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:33.391566    2583 generic.go:356] "Generic (PLEG): container finished" podID="d01de4d9-e0fd-4544-8592-bd5c9d1c2233" containerID="a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1" exitCode=0
Jul 28 18:08:33.392008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:33.391614    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ffxk7"
Jul 28 18:08:33.392008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:33.391652    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ffxk7" event={"ID":"d01de4d9-e0fd-4544-8592-bd5c9d1c2233","Type":"ContainerDied","Data":"a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1"}
Jul 28 18:08:33.392008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:33.391702    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ffxk7" event={"ID":"d01de4d9-e0fd-4544-8592-bd5c9d1c2233","Type":"ContainerDied","Data":"d54020ddeff9bd61ee828a81ac2879ed3646493e98b3de80458ab98381f2b97e"}
Jul 28 18:08:33.392008 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:33.391727    2583 scope.go:117] "RemoveContainer" containerID="a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1"
Jul 28 18:08:33.401148 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:33.401132    2583 scope.go:117] "RemoveContainer" containerID="a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1"
Jul 28 18:08:33.401403 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:08:33.401385    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1\": container with ID starting with a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1 not found: ID does not exist" containerID="a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1"
Jul 28 18:08:33.401480 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:33.401410    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1"} err="failed to get container status \"a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1\": rpc error: code = NotFound desc = could not find container \"a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1\": container with ID starting with a867334e998713d3b3c331822c29e9f3597694f87965867ed9f93eab955090d1 not found: ID does not exist"
Jul 28 18:08:33.414404 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:33.414377    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ffxk7"]
Jul 28 18:08:33.418502 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:33.418480    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ffxk7"]
Jul 28 18:08:34.822659 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:34.822627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:34.926006 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:34.925972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:35.404605 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:35.404569    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d01de4d9-e0fd-4544-8592-bd5c9d1c2233" path="/var/lib/kubelet/pods/d01de4d9-e0fd-4544-8592-bd5c9d1c2233/volumes"
Jul 28 18:08:40.168736 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:40.168699    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-878fw"
Jul 28 18:08:40.169237 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:40.168771    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-878fw"
Jul 28 18:08:40.190056 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:40.190031    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-878fw"
Jul 28 18:08:40.442241 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:40.442210    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-878fw"
Jul 28 18:08:55.713006 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:55.712966    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:55.780190 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:55.780150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:58.615268 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:58.615232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:59.742489 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:59.742447    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:59.841052 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:59.841014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:59.945701 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:08:59.945670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:00.047451 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:00.047350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:00.478130 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:00.478098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:01.250803 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.250766    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x"]
Jul 28 18:09:01.251181 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.251104    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01de4d9-e0fd-4544-8592-bd5c9d1c2233" containerName="registry-server"
Jul 28 18:09:01.251181 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.251115    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01de4d9-e0fd-4544-8592-bd5c9d1c2233" containerName="registry-server"
Jul 28 18:09:01.251253 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.251198    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d01de4d9-e0fd-4544-8592-bd5c9d1c2233" containerName="registry-server"
Jul 28 18:09:01.255425 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.255403    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x"
Jul 28 18:09:01.258609 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.258582    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 18:09:01.258745 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.258730    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7jdw2\""
Jul 28 18:09:01.268333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.268311    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x"]
Jul 28 18:09:01.322314 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.322277    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4tmm\" (UniqueName: \"kubernetes.io/projected/c77998b7-160c-40bd-8b54-61ee03492679-kube-api-access-g4tmm\") pod \"dns-operator-controller-manager-864dd78c4c-8k25x\" (UID: \"c77998b7-160c-40bd-8b54-61ee03492679\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x"
Jul 28 18:09:01.422935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.422902    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4tmm\" (UniqueName: \"kubernetes.io/projected/c77998b7-160c-40bd-8b54-61ee03492679-kube-api-access-g4tmm\") pod \"dns-operator-controller-manager-864dd78c4c-8k25x\" (UID: \"c77998b7-160c-40bd-8b54-61ee03492679\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x"
Jul 28 18:09:01.432228 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.432203    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4tmm\" (UniqueName: \"kubernetes.io/projected/c77998b7-160c-40bd-8b54-61ee03492679-kube-api-access-g4tmm\") pod \"dns-operator-controller-manager-864dd78c4c-8k25x\" (UID: \"c77998b7-160c-40bd-8b54-61ee03492679\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x"
Jul 28 18:09:01.565584 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.565500    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x"
Jul 28 18:09:01.697533 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:01.697501    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x"]
Jul 28 18:09:01.701999 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:09:01.701971    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc77998b7_160c_40bd_8b54_61ee03492679.slice/crio-bb300e4a8100c0199e3151afb6402f53f477dbd4a187e54334af31d4126ed3b7 WatchSource:0}: Error finding container bb300e4a8100c0199e3151afb6402f53f477dbd4a187e54334af31d4126ed3b7: Status 404 returned error can't find the container with id bb300e4a8100c0199e3151afb6402f53f477dbd4a187e54334af31d4126ed3b7
Jul 28 18:09:02.014168 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:02.014131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:02.520018 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:02.519974    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x" event={"ID":"c77998b7-160c-40bd-8b54-61ee03492679","Type":"ContainerStarted","Data":"bb300e4a8100c0199e3151afb6402f53f477dbd4a187e54334af31d4126ed3b7"}
Jul 28 18:09:02.855336 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:02.855241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:04.529461 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:04.529429    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x" event={"ID":"c77998b7-160c-40bd-8b54-61ee03492679","Type":"ContainerStarted","Data":"1d49cffa754f466128b9b1facb09375f765f1baf550d4f1a5ae8f48b6da52de8"}
Jul 28 18:09:04.529899 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:04.529540    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x"
Jul 28 18:09:04.547832 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:04.547767    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x" podStartSLOduration=1.3052538390000001 podStartE2EDuration="3.547752139s" totalImagesPullingTime="2.2424983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:09:04.545923868 +0000 UTC m=+495.622485864" watchObservedRunningTime="2026-07-28 18:09:04.547752139 +0000 UTC m=+495.624314153"
Jul 28 18:09:05.361525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:05.361489    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2"]
Jul 28 18:09:05.364938 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:05.364922    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2"
Jul 28 18:09:05.369164 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:05.369142    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-k8tfx\""
Jul 28 18:09:05.374705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:05.374682    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2"]
Jul 28 18:09:05.457759 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:05.457722    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tshx9\" (UniqueName: \"kubernetes.io/projected/3e52f160-84d4-419d-a243-a31a003fdbd2-kube-api-access-tshx9\") pod \"authorino-operator-75cc8d5cb7-nwqx2\" (UID: \"3e52f160-84d4-419d-a243-a31a003fdbd2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2"
Jul 28 18:09:05.558191 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:05.558155    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tshx9\" (UniqueName: \"kubernetes.io/projected/3e52f160-84d4-419d-a243-a31a003fdbd2-kube-api-access-tshx9\") pod \"authorino-operator-75cc8d5cb7-nwqx2\" (UID: \"3e52f160-84d4-419d-a243-a31a003fdbd2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2"
Jul 28 18:09:05.568192 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:05.568158    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tshx9\" (UniqueName: \"kubernetes.io/projected/3e52f160-84d4-419d-a243-a31a003fdbd2-kube-api-access-tshx9\") pod \"authorino-operator-75cc8d5cb7-nwqx2\" (UID: \"3e52f160-84d4-419d-a243-a31a003fdbd2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2"
Jul 28 18:09:05.675335 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:05.675245    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2"
Jul 28 18:09:05.814272 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:05.814235    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2"]
Jul 28 18:09:05.817689 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:09:05.817657    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e52f160_84d4_419d_a243_a31a003fdbd2.slice/crio-f4a3f012513c57f3d6d58d9e7ab27748523c6a46ab02f963c611a61edc9943d1 WatchSource:0}: Error finding container f4a3f012513c57f3d6d58d9e7ab27748523c6a46ab02f963c611a61edc9943d1: Status 404 returned error can't find the container with id f4a3f012513c57f3d6d58d9e7ab27748523c6a46ab02f963c611a61edc9943d1
Jul 28 18:09:06.036873 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:06.036845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:06.487726 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:06.487692    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:06.541098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:06.541059    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2" event={"ID":"3e52f160-84d4-419d-a243-a31a003fdbd2","Type":"ContainerStarted","Data":"f4a3f012513c57f3d6d58d9e7ab27748523c6a46ab02f963c611a61edc9943d1"}
Jul 28 18:09:06.589619 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:06.589452    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:08.435492 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:08.435387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:08.552404 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:08.552365    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2" event={"ID":"3e52f160-84d4-419d-a243-a31a003fdbd2","Type":"ContainerStarted","Data":"0918035a9fd3c6091c3b86a79159654bb16e777869dd6596473d83bb79b36cba"}
Jul 28 18:09:08.552582 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:08.552449    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2"
Jul 28 18:09:08.573430 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:08.573382    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2" podStartSLOduration=1.425831824 podStartE2EDuration="3.573368299s" totalImagesPullingTime="2.147536475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:09:08.571635719 +0000 UTC m=+499.648197715" watchObservedRunningTime="2026-07-28 18:09:08.573368299 +0000 UTC m=+499.649930296"
Jul 28 18:09:09.647342 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:09.647307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:10.210603 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.210565    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"]
Jul 28 18:09:10.214428 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.214394    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:10.220432 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.220410    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mtdpn\""
Jul 28 18:09:10.237834 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.237790    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"]
Jul 28 18:09:10.253764 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.253725    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:10.308430 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.308397    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkdt4\" (UniqueName: \"kubernetes.io/projected/8857ee1e-2c8e-4b8c-9b85-352ef4000499-kube-api-access-jkdt4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" (UID: \"8857ee1e-2c8e-4b8c-9b85-352ef4000499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:10.308592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.308436    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8857ee1e-2c8e-4b8c-9b85-352ef4000499-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" (UID: \"8857ee1e-2c8e-4b8c-9b85-352ef4000499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:10.409203 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.409167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkdt4\" (UniqueName: \"kubernetes.io/projected/8857ee1e-2c8e-4b8c-9b85-352ef4000499-kube-api-access-jkdt4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" (UID: \"8857ee1e-2c8e-4b8c-9b85-352ef4000499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:10.409370 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.409213    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8857ee1e-2c8e-4b8c-9b85-352ef4000499-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" (UID: \"8857ee1e-2c8e-4b8c-9b85-352ef4000499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:10.409612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.409594    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8857ee1e-2c8e-4b8c-9b85-352ef4000499-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" (UID: \"8857ee1e-2c8e-4b8c-9b85-352ef4000499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:10.419146 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.419116    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkdt4\" (UniqueName: \"kubernetes.io/projected/8857ee1e-2c8e-4b8c-9b85-352ef4000499-kube-api-access-jkdt4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" (UID: \"8857ee1e-2c8e-4b8c-9b85-352ef4000499\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:10.419292 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.419183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:10.524414 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.524338    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:10.660028 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:10.660002    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"]
Jul 28 18:09:10.662313 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:09:10.662279    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8857ee1e_2c8e_4b8c_9b85_352ef4000499.slice/crio-9baa10c69a51dcdaf46d9d0931859d1fb2b2f5b6b9dcef3fc369b93046841c3a WatchSource:0}: Error finding container 9baa10c69a51dcdaf46d9d0931859d1fb2b2f5b6b9dcef3fc369b93046841c3a: Status 404 returned error can't find the container with id 9baa10c69a51dcdaf46d9d0931859d1fb2b2f5b6b9dcef3fc369b93046841c3a
Jul 28 18:09:11.218529 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:11.218499    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:11.445529 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:11.445492    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:11.569146 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:11.569071    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq" event={"ID":"8857ee1e-2c8e-4b8c-9b85-352ef4000499","Type":"ContainerStarted","Data":"9baa10c69a51dcdaf46d9d0931859d1fb2b2f5b6b9dcef3fc369b93046841c3a"}
Jul 28 18:09:14.429752 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:14.429716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:14.534718 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:14.534671    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:15.536770 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:15.536717    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8k25x"
Jul 28 18:09:16.574881 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:16.574839    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:16.675526 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:16.675488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:18.422080 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:18.422016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:18.560086 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:18.560045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:19.559032 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:19.559006    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwqx2"
Jul 28 18:09:19.612069 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:19.612039    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq" event={"ID":"8857ee1e-2c8e-4b8c-9b85-352ef4000499","Type":"ContainerStarted","Data":"6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39"}
Jul 28 18:09:19.612226 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:19.612104    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:19.632284 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:19.632233    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq" podStartSLOduration=1.6752894139999999 podStartE2EDuration="9.632213472s" totalImagesPullingTime="7.956924058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:09:19.632097459 +0000 UTC m=+510.708659459" watchObservedRunningTime="2026-07-28 18:09:19.632213472 +0000 UTC m=+510.708775469"
Jul 28 18:09:19.832229 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:19.832104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:19.934456 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:19.934424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:20.279077 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:20.279034    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:20.372197 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:20.372153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:21.277129 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:21.277092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:21.482159 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:21.482122    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:23.817760 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:23.817727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:23.886037 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:23.885999    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:27.686046 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:27.686004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:27.786230 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:27.786199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:30.247317 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:30.247281    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:30.348541 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:30.348507    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:30.618579 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:30.618500    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:31.739087 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:31.739053    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"]
Jul 28 18:09:31.742485 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:31.742468    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:31.766253 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:31.766217    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"]
Jul 28 18:09:31.908508 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:31.908457    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m7qq\" (UniqueName: \"kubernetes.io/projected/71bd0a49-5672-4832-8ab1-5f8fc01682f6-kube-api-access-6m7qq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" (UID: \"71bd0a49-5672-4832-8ab1-5f8fc01682f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:31.908681 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:31.908626    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71bd0a49-5672-4832-8ab1-5f8fc01682f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" (UID: \"71bd0a49-5672-4832-8ab1-5f8fc01682f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:32.009645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.009552    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71bd0a49-5672-4832-8ab1-5f8fc01682f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" (UID: \"71bd0a49-5672-4832-8ab1-5f8fc01682f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:32.009645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.009609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m7qq\" (UniqueName: \"kubernetes.io/projected/71bd0a49-5672-4832-8ab1-5f8fc01682f6-kube-api-access-6m7qq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" (UID: \"71bd0a49-5672-4832-8ab1-5f8fc01682f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:32.010067 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.010042    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71bd0a49-5672-4832-8ab1-5f8fc01682f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" (UID: \"71bd0a49-5672-4832-8ab1-5f8fc01682f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:32.018774 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.018749    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m7qq\" (UniqueName: \"kubernetes.io/projected/71bd0a49-5672-4832-8ab1-5f8fc01682f6-kube-api-access-6m7qq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" (UID: \"71bd0a49-5672-4832-8ab1-5f8fc01682f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:32.052210 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.052180    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:32.191434 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.191409    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"]
Jul 28 18:09:32.193549 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:09:32.193522    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71bd0a49_5672_4832_8ab1_5f8fc01682f6.slice/crio-1f6d1ebe7ef251dca87c85ac008ea1ccbce5bb166479d5a0c7a018434992cdd6 WatchSource:0}: Error finding container 1f6d1ebe7ef251dca87c85ac008ea1ccbce5bb166479d5a0c7a018434992cdd6: Status 404 returned error can't find the container with id 1f6d1ebe7ef251dca87c85ac008ea1ccbce5bb166479d5a0c7a018434992cdd6
Jul 28 18:09:32.405313 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.405281    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"]
Jul 28 18:09:32.410516 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.410491    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"]
Jul 28 18:09:32.420479 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.420455    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"]
Jul 28 18:09:32.420724 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.420678    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" containerName="manager" containerID="cri-o://6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39" gracePeriod=2
Jul 28 18:09:32.434191 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.434129    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"]
Jul 28 18:09:32.435988 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.435963    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"]
Jul 28 18:09:32.436348 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.436330    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" containerName="manager"
Jul 28 18:09:32.436422 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.436351    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" containerName="manager"
Jul 28 18:09:32.436422 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.436374    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" containerName="manager"
Jul 28 18:09:32.436422 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.436382    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" containerName="manager"
Jul 28 18:09:32.436521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.436479    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" containerName="manager"
Jul 28 18:09:32.436521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.436490    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" containerName="manager"
Jul 28 18:09:32.442085 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.442069    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:32.444889 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:32.444856    2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"8857ee1e-2c8e-4b8c-9b85-352ef4000499\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-9shkq\": pods \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:32.453684 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.453638    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"]
Jul 28 18:09:32.459896 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.459874    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"]
Jul 28 18:09:32.463471 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.463450    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:32.468588 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:32.468555    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:32.473409 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:32.473385    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:32.485585 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.485563    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"]
Jul 28 18:09:32.615429 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.615399    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-498wj\" (UniqueName: \"kubernetes.io/projected/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-kube-api-access-498wj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sk7xs\" (UID: \"5a6360ce-2cfc-483a-abc5-ae7e84fd338d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:32.615591 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.615469    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d8eae90-42a3-4ddb-b2e6-2bac46174854-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dj6j6\" (UID: \"1d8eae90-42a3-4ddb-b2e6-2bac46174854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:32.615591 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.615540    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkxft\" (UniqueName: \"kubernetes.io/projected/1d8eae90-42a3-4ddb-b2e6-2bac46174854-kube-api-access-vkxft\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dj6j6\" (UID: \"1d8eae90-42a3-4ddb-b2e6-2bac46174854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:32.615701 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.615590    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sk7xs\" (UID: \"5a6360ce-2cfc-483a-abc5-ae7e84fd338d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:32.651736 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.651714    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:32.654675 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:32.654650    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:32.671545 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.671521    2583 generic.go:356] "Generic (PLEG): container finished" podID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" containerID="6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39" exitCode=0
Jul 28 18:09:32.671649 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.671566    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:32.671649 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.671636    2583 scope.go:117] "RemoveContainer" containerID="6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39"
Jul 28 18:09:32.673619 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:32.673597    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q" container="manager"
Jul 28 18:09:32.674459 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:32.674425    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:32.676853 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:32.676810    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:32.679199 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:32.679173    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:32.679942 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.679921    2583 scope.go:117] "RemoveContainer" containerID="6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39"
Jul 28 18:09:32.680216 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:32.680190    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39\": container with ID starting with 6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39 not found: ID does not exist" containerID="6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39"
Jul 28 18:09:32.680293 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.680225    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39"} err="failed to get container status \"6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39\": rpc error: code = NotFound desc = could not find container \"6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39\": container with ID starting with 6c070cacad19b8f7a805d7b6f3dfac3f36bfdb147b67e19cbb0a58e7a5ce2a39 not found: ID does not exist"
Jul 28 18:09:32.716233 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.716210    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkxft\" (UniqueName: \"kubernetes.io/projected/1d8eae90-42a3-4ddb-b2e6-2bac46174854-kube-api-access-vkxft\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dj6j6\" (UID: \"1d8eae90-42a3-4ddb-b2e6-2bac46174854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:32.716361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.716254    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sk7xs\" (UID: \"5a6360ce-2cfc-483a-abc5-ae7e84fd338d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:32.716361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.716286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498wj\" (UniqueName: \"kubernetes.io/projected/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-kube-api-access-498wj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sk7xs\" (UID: \"5a6360ce-2cfc-483a-abc5-ae7e84fd338d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:32.716361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.716318    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d8eae90-42a3-4ddb-b2e6-2bac46174854-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dj6j6\" (UID: \"1d8eae90-42a3-4ddb-b2e6-2bac46174854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:32.716612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.716593    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sk7xs\" (UID: \"5a6360ce-2cfc-483a-abc5-ae7e84fd338d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:32.716663 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.716649    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d8eae90-42a3-4ddb-b2e6-2bac46174854-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dj6j6\" (UID: \"1d8eae90-42a3-4ddb-b2e6-2bac46174854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:32.734365 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.734338    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkxft\" (UniqueName: \"kubernetes.io/projected/1d8eae90-42a3-4ddb-b2e6-2bac46174854-kube-api-access-vkxft\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dj6j6\" (UID: \"1d8eae90-42a3-4ddb-b2e6-2bac46174854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:32.734741 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.734725    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-498wj\" (UniqueName: \"kubernetes.io/projected/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-kube-api-access-498wj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sk7xs\" (UID: \"5a6360ce-2cfc-483a-abc5-ae7e84fd338d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:32.779566 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.779538    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:32.787577 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.787554    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:32.817247 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.817218    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkdt4\" (UniqueName: \"kubernetes.io/projected/8857ee1e-2c8e-4b8c-9b85-352ef4000499-kube-api-access-jkdt4\") pod \"8857ee1e-2c8e-4b8c-9b85-352ef4000499\" (UID: \"8857ee1e-2c8e-4b8c-9b85-352ef4000499\") "
Jul 28 18:09:32.817432 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.817360    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8857ee1e-2c8e-4b8c-9b85-352ef4000499-extensions-socket-volume\") pod \"8857ee1e-2c8e-4b8c-9b85-352ef4000499\" (UID: \"8857ee1e-2c8e-4b8c-9b85-352ef4000499\") "
Jul 28 18:09:32.817948 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.817904    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8857ee1e-2c8e-4b8c-9b85-352ef4000499-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8857ee1e-2c8e-4b8c-9b85-352ef4000499" (UID: "8857ee1e-2c8e-4b8c-9b85-352ef4000499"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:09:32.819343 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.819322    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8857ee1e-2c8e-4b8c-9b85-352ef4000499-kube-api-access-jkdt4" (OuterVolumeSpecName: "kube-api-access-jkdt4") pod "8857ee1e-2c8e-4b8c-9b85-352ef4000499" (UID: "8857ee1e-2c8e-4b8c-9b85-352ef4000499"). InnerVolumeSpecName "kube-api-access-jkdt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:09:32.864137 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.863652    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:32.918527 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.918497    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkdt4\" (UniqueName: \"kubernetes.io/projected/8857ee1e-2c8e-4b8c-9b85-352ef4000499-kube-api-access-jkdt4\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:09:32.918527 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.918526    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8857ee1e-2c8e-4b8c-9b85-352ef4000499-extensions-socket-volume\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:09:32.953778 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.953738    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"]
Jul 28 18:09:32.957296 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:09:32.957270    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8eae90_42a3_4ddb_b2e6_2bac46174854.slice/crio-8691707234a10fd5d1dbb6190690c101c12aa16812009fe253f94fcf1cf7ecdf WatchSource:0}: Error finding container 8691707234a10fd5d1dbb6190690c101c12aa16812009fe253f94fcf1cf7ecdf: Status 404 returned error can't find the container with id 8691707234a10fd5d1dbb6190690c101c12aa16812009fe253f94fcf1cf7ecdf
Jul 28 18:09:32.973896 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.972127    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:32.978828 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:32.978795    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"]
Jul 28 18:09:32.980916 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:09:32.980890    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a6360ce_2cfc_483a_abc5_ae7e84fd338d.slice/crio-2b5875ce2a5a003678baefc6d844278412372b470e14f3d372142591e2c5830e WatchSource:0}: Error finding container 2b5875ce2a5a003678baefc6d844278412372b470e14f3d372142591e2c5830e: Status 404 returned error can't find the container with id 2b5875ce2a5a003678baefc6d844278412372b470e14f3d372142591e2c5830e
Jul 28 18:09:33.001518 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:33.001488    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9shkq\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9shkq"
Jul 28 18:09:33.004296 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:33.004266    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:33.404939 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:33.404910    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8857ee1e-2c8e-4b8c-9b85-352ef4000499" path="/var/lib/kubelet/pods/8857ee1e-2c8e-4b8c-9b85-352ef4000499/volumes"
Jul 28 18:09:33.680333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:33.680239    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6" event={"ID":"1d8eae90-42a3-4ddb-b2e6-2bac46174854","Type":"ContainerStarted","Data":"51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197"}
Jul 28 18:09:33.680333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:33.680295    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6" event={"ID":"1d8eae90-42a3-4ddb-b2e6-2bac46174854","Type":"ContainerStarted","Data":"8691707234a10fd5d1dbb6190690c101c12aa16812009fe253f94fcf1cf7ecdf"}
Jul 28 18:09:33.680556 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:33.680410    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:33.682600 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:33.682573    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q" podUID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" containerName="manager" containerID="cri-o://8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226" gracePeriod=2
Jul 28 18:09:33.684148 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:33.684120    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs" event={"ID":"5a6360ce-2cfc-483a-abc5-ae7e84fd338d","Type":"ContainerStarted","Data":"aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511"}
Jul 28 18:09:33.684288 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:33.684183    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:33.684288 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:33.684210    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs" event={"ID":"5a6360ce-2cfc-483a-abc5-ae7e84fd338d","Type":"ContainerStarted","Data":"2b5875ce2a5a003678baefc6d844278412372b470e14f3d372142591e2c5830e"}
Jul 28 18:09:33.705712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:33.705671    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6" podStartSLOduration=1.705658058 podStartE2EDuration="1.705658058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:09:33.702803084 +0000 UTC m=+524.779365276" watchObservedRunningTime="2026-07-28 18:09:33.705658058 +0000 UTC m=+524.782220055"
Jul 28 18:09:33.928184 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:33.928152    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:33.931080 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:33.931009    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:34.027872 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.027809    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m7qq\" (UniqueName: \"kubernetes.io/projected/71bd0a49-5672-4832-8ab1-5f8fc01682f6-kube-api-access-6m7qq\") pod \"71bd0a49-5672-4832-8ab1-5f8fc01682f6\" (UID: \"71bd0a49-5672-4832-8ab1-5f8fc01682f6\") "
Jul 28 18:09:34.028073 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.027967    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71bd0a49-5672-4832-8ab1-5f8fc01682f6-extensions-socket-volume\") pod \"71bd0a49-5672-4832-8ab1-5f8fc01682f6\" (UID: \"71bd0a49-5672-4832-8ab1-5f8fc01682f6\") "
Jul 28 18:09:34.028238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.028211    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71bd0a49-5672-4832-8ab1-5f8fc01682f6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "71bd0a49-5672-4832-8ab1-5f8fc01682f6" (UID: "71bd0a49-5672-4832-8ab1-5f8fc01682f6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:09:34.029896 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.029875    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71bd0a49-5672-4832-8ab1-5f8fc01682f6-kube-api-access-6m7qq" (OuterVolumeSpecName: "kube-api-access-6m7qq") pod "71bd0a49-5672-4832-8ab1-5f8fc01682f6" (UID: "71bd0a49-5672-4832-8ab1-5f8fc01682f6"). InnerVolumeSpecName "kube-api-access-6m7qq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:09:34.129546 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.129511    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71bd0a49-5672-4832-8ab1-5f8fc01682f6-extensions-socket-volume\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:09:34.129546 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.129541    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m7qq\" (UniqueName: \"kubernetes.io/projected/71bd0a49-5672-4832-8ab1-5f8fc01682f6-kube-api-access-6m7qq\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:09:34.690103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.690065    2583 generic.go:356] "Generic (PLEG): container finished" podID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" containerID="8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226" exitCode=2
Jul 28 18:09:34.690257 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.690117    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:34.690257 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.690125    2583 scope.go:117] "RemoveContainer" containerID="8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226"
Jul 28 18:09:34.693394 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:34.693369    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:34.698582 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.698564    2583 scope.go:117] "RemoveContainer" containerID="8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226"
Jul 28 18:09:34.698863 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:34.698841    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226\": container with ID starting with 8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226 not found: ID does not exist" containerID="8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226"
Jul 28 18:09:34.698915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:34.698873    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226"} err="failed to get container status \"8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226\": rpc error: code = NotFound desc = could not find container \"8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226\": container with ID starting with 8f21c458970c010a1a1bddbb8aea2b5dbd0dcf61e1b83ab5390836e1ac2a1226 not found: ID does not exist"
Jul 28 18:09:34.700731 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:34.700711    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-tqr6q\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-tqr6q"
Jul 28 18:09:35.405284 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:35.405250    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71bd0a49-5672-4832-8ab1-5f8fc01682f6" path="/var/lib/kubelet/pods/71bd0a49-5672-4832-8ab1-5f8fc01682f6/volumes"
Jul 28 18:09:38.242504 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:38.242460    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:38.343061 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:38.343028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:38.645607 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:38.645520    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:38.745183 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:38.745152    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:42.247533 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:42.247460    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:42.349999 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:42.349961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:44.692680 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:44.692648    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:44.693137 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:44.692841    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:44.717539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:44.717486    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs" podStartSLOduration=12.717472617 podStartE2EDuration="12.717472617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:09:33.734024045 +0000 UTC m=+524.810586044" watchObservedRunningTime="2026-07-28 18:09:44.717472617 +0000 UTC m=+535.794034648"
Jul 28 18:09:44.771896 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:44.771864    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"]
Jul 28 18:09:44.772103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:44.772078    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs" podUID="5a6360ce-2cfc-483a-abc5-ae7e84fd338d" containerName="manager" containerID="cri-o://aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511" gracePeriod=10
Jul 28 18:09:45.022879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.022855    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:45.121637 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.121608    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-498wj\" (UniqueName: \"kubernetes.io/projected/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-kube-api-access-498wj\") pod \"5a6360ce-2cfc-483a-abc5-ae7e84fd338d\" (UID: \"5a6360ce-2cfc-483a-abc5-ae7e84fd338d\") "
Jul 28 18:09:45.121842 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.121698    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-extensions-socket-volume\") pod \"5a6360ce-2cfc-483a-abc5-ae7e84fd338d\" (UID: \"5a6360ce-2cfc-483a-abc5-ae7e84fd338d\") "
Jul 28 18:09:45.122084 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.122060    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5a6360ce-2cfc-483a-abc5-ae7e84fd338d" (UID: "5a6360ce-2cfc-483a-abc5-ae7e84fd338d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:09:45.123645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.123624    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-kube-api-access-498wj" (OuterVolumeSpecName: "kube-api-access-498wj") pod "5a6360ce-2cfc-483a-abc5-ae7e84fd338d" (UID: "5a6360ce-2cfc-483a-abc5-ae7e84fd338d"). InnerVolumeSpecName "kube-api-access-498wj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:09:45.153604 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.153578    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"]
Jul 28 18:09:45.154003 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.153988    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a6360ce-2cfc-483a-abc5-ae7e84fd338d" containerName="manager"
Jul 28 18:09:45.154059 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.154005    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a6360ce-2cfc-483a-abc5-ae7e84fd338d" containerName="manager"
Jul 28 18:09:45.154745 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.154716    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a6360ce-2cfc-483a-abc5-ae7e84fd338d" containerName="manager"
Jul 28 18:09:45.159950 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.157674    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:09:45.175112 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.175085    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"]
Jul 28 18:09:45.222954 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.222918    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk8lx\" (UniqueName: \"kubernetes.io/projected/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-kube-api-access-xk8lx\") pod \"kuadrant-operator-controller-manager-86878c769f-bbxc9\" (UID: \"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:09:45.222954 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.222954    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bbxc9\" (UID: \"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:09:45.223205 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.223104    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-extensions-socket-volume\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:09:45.223205 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.223128    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-498wj\" (UniqueName: \"kubernetes.io/projected/5a6360ce-2cfc-483a-abc5-ae7e84fd338d-kube-api-access-498wj\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:09:45.324149 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.324050    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk8lx\" (UniqueName: \"kubernetes.io/projected/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-kube-api-access-xk8lx\") pod \"kuadrant-operator-controller-manager-86878c769f-bbxc9\" (UID: \"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:09:45.324149 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.324094    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bbxc9\" (UID: \"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:09:45.324505 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.324482    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bbxc9\" (UID: \"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:09:45.334841 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.334805    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk8lx\" (UniqueName: \"kubernetes.io/projected/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-kube-api-access-xk8lx\") pod \"kuadrant-operator-controller-manager-86878c769f-bbxc9\" (UID: \"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:09:45.470242 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.470207    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:09:45.632002 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.631978    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"]
Jul 28 18:09:45.634311 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:09:45.634286    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d9fa15_b0c1_4264_a08d_7e0ebd8861b9.slice/crio-eb0cd25eb65b5baa19b1afb5dcf175a7b73f9784b098a86c2da0a5d1e396bc0b WatchSource:0}: Error finding container eb0cd25eb65b5baa19b1afb5dcf175a7b73f9784b098a86c2da0a5d1e396bc0b: Status 404 returned error can't find the container with id eb0cd25eb65b5baa19b1afb5dcf175a7b73f9784b098a86c2da0a5d1e396bc0b
Jul 28 18:09:45.748065 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.748024    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9" event={"ID":"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9","Type":"ContainerStarted","Data":"b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794"}
Jul 28 18:09:45.748525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.748091    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9" event={"ID":"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9","Type":"ContainerStarted","Data":"eb0cd25eb65b5baa19b1afb5dcf175a7b73f9784b098a86c2da0a5d1e396bc0b"}
Jul 28 18:09:45.748525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.748221    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:09:45.749311 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.749289    2583 generic.go:356] "Generic (PLEG): container finished" podID="5a6360ce-2cfc-483a-abc5-ae7e84fd338d" containerID="aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511" exitCode=0
Jul 28 18:09:45.749429 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.749340    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs" event={"ID":"5a6360ce-2cfc-483a-abc5-ae7e84fd338d","Type":"ContainerDied","Data":"aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511"}
Jul 28 18:09:45.749429 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.749345    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"
Jul 28 18:09:45.749429 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.749371    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs" event={"ID":"5a6360ce-2cfc-483a-abc5-ae7e84fd338d","Type":"ContainerDied","Data":"2b5875ce2a5a003678baefc6d844278412372b470e14f3d372142591e2c5830e"}
Jul 28 18:09:45.749429 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.749392    2583 scope.go:117] "RemoveContainer" containerID="aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511"
Jul 28 18:09:45.758518 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.758499    2583 scope.go:117] "RemoveContainer" containerID="aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511"
Jul 28 18:09:45.758774 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:45.758754    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511\": container with ID starting with aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511 not found: ID does not exist" containerID="aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511"
Jul 28 18:09:45.758839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.758786    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511"} err="failed to get container status \"aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511\": rpc error: code = NotFound desc = could not find container \"aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511\": container with ID starting with aca719e284b5621217adc8a68c35af7db531147adbbd87b198e30b5326802511 not found: ID does not exist"
Jul 28 18:09:45.769943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.769900    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9" podStartSLOduration=0.769887128 podStartE2EDuration="769.887128ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:09:45.768619673 +0000 UTC m=+536.845181670" watchObservedRunningTime="2026-07-28 18:09:45.769887128 +0000 UTC m=+536.846449125"
Jul 28 18:09:45.790873 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.790847    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"]
Jul 28 18:09:45.797733 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:45.797706    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sk7xs"]
Jul 28 18:09:47.405810 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:47.405774    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a6360ce-2cfc-483a-abc5-ae7e84fd338d" path="/var/lib/kubelet/pods/5a6360ce-2cfc-483a-abc5-ae7e84fd338d/volumes"
Jul 28 18:09:48.742723 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:48.742685    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:48.841155 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:48.841116    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:56.757641 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:56.757606    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:09:56.830554 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:56.830522    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"]
Jul 28 18:09:56.830753 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:56.830732    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6" podUID="1d8eae90-42a3-4ddb-b2e6-2bac46174854" containerName="manager" containerID="cri-o://51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197" gracePeriod=10
Jul 28 18:09:57.081853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.081809    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:57.129975 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.129942    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkxft\" (UniqueName: \"kubernetes.io/projected/1d8eae90-42a3-4ddb-b2e6-2bac46174854-kube-api-access-vkxft\") pod \"1d8eae90-42a3-4ddb-b2e6-2bac46174854\" (UID: \"1d8eae90-42a3-4ddb-b2e6-2bac46174854\") "
Jul 28 18:09:57.130141 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.129992    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d8eae90-42a3-4ddb-b2e6-2bac46174854-extensions-socket-volume\") pod \"1d8eae90-42a3-4ddb-b2e6-2bac46174854\" (UID: \"1d8eae90-42a3-4ddb-b2e6-2bac46174854\") "
Jul 28 18:09:57.130433 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.130391    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d8eae90-42a3-4ddb-b2e6-2bac46174854-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1d8eae90-42a3-4ddb-b2e6-2bac46174854" (UID: "1d8eae90-42a3-4ddb-b2e6-2bac46174854"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:09:57.132113 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.132085    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d8eae90-42a3-4ddb-b2e6-2bac46174854-kube-api-access-vkxft" (OuterVolumeSpecName: "kube-api-access-vkxft") pod "1d8eae90-42a3-4ddb-b2e6-2bac46174854" (UID: "1d8eae90-42a3-4ddb-b2e6-2bac46174854"). InnerVolumeSpecName "kube-api-access-vkxft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:09:57.231094 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.231057    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkxft\" (UniqueName: \"kubernetes.io/projected/1d8eae90-42a3-4ddb-b2e6-2bac46174854-kube-api-access-vkxft\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:09:57.231094 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.231086    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d8eae90-42a3-4ddb-b2e6-2bac46174854-extensions-socket-volume\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:09:57.807318 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.807284    2583 generic.go:356] "Generic (PLEG): container finished" podID="1d8eae90-42a3-4ddb-b2e6-2bac46174854" containerID="51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197" exitCode=0
Jul 28 18:09:57.807768 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.807348    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"
Jul 28 18:09:57.807768 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.807378    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6" event={"ID":"1d8eae90-42a3-4ddb-b2e6-2bac46174854","Type":"ContainerDied","Data":"51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197"}
Jul 28 18:09:57.807768 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.807450    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6" event={"ID":"1d8eae90-42a3-4ddb-b2e6-2bac46174854","Type":"ContainerDied","Data":"8691707234a10fd5d1dbb6190690c101c12aa16812009fe253f94fcf1cf7ecdf"}
Jul 28 18:09:57.807768 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.807484    2583 scope.go:117] "RemoveContainer" containerID="51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197"
Jul 28 18:09:57.815762 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.815747    2583 scope.go:117] "RemoveContainer" containerID="51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197"
Jul 28 18:09:57.816033 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:09:57.816012    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197\": container with ID starting with 51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197 not found: ID does not exist" containerID="51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197"
Jul 28 18:09:57.816100 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.816039    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197"} err="failed to get container status \"51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197\": rpc error: code = NotFound desc = could not find container \"51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197\": container with ID starting with 51d27813e627a7dcc065145e33b006b6467cd4f52f3910273dbc22b11340d197 not found: ID does not exist"
Jul 28 18:09:57.831116 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.831091    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"]
Jul 28 18:09:57.838990 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:57.838966    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dj6j6"]
Jul 28 18:09:59.405326 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:09:59.405292    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d8eae90-42a3-4ddb-b2e6-2bac46174854" path="/var/lib/kubelet/pods/1d8eae90-42a3-4ddb-b2e6-2bac46174854/volumes"
Jul 28 18:10:01.121488 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.121450    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"]
Jul 28 18:10:01.122012 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.121891    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d8eae90-42a3-4ddb-b2e6-2bac46174854" containerName="manager"
Jul 28 18:10:01.122012 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.121906    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d8eae90-42a3-4ddb-b2e6-2bac46174854" containerName="manager"
Jul 28 18:10:01.122012 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.121998    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d8eae90-42a3-4ddb-b2e6-2bac46174854" containerName="manager"
Jul 28 18:10:01.125289 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.125266    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.128228 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.128207    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rrv2m\""
Jul 28 18:10:01.137251 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.137229    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"]
Jul 28 18:10:01.167724 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.167683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.167921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.167747    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.167921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.167838    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.167921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.167896    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.167921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.167914    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.168090 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.167949    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.168090 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.168007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.168090 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.168043    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.168090 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.168061    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kshc8\" (UniqueName: \"kubernetes.io/projected/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-kube-api-access-kshc8\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.208104 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.208066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:01.269152 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269120    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269345 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269174    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269345 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269219    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269345 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269243    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269345 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269282    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269345 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269345 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269640 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269488    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kshc8\" (UniqueName: \"kubernetes.io/projected/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-kube-api-access-kshc8\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269640 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269540    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269747 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269727    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269940 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269899    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.269981 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.269923    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.270043 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.270025    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.271740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.271719    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.271973 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.271954    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.278894 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.278868    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.278894 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.278883    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kshc8\" (UniqueName: \"kubernetes.io/projected/ea3a4c2a-2d05-4ce3-8a73-5fbcae566368-kube-api-access-kshc8\") pod \"maas-default-gateway-openshift-default-5ccb968c66-jlmbq\" (UID: \"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.317991 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.317955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:01.435861 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.435764    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:01.576325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.576299    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"]
Jul 28 18:10:01.578062 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:10:01.578034    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3a4c2a_2d05_4ce3_8a73_5fbcae566368.slice/crio-3d3f758972c1e0c377358ff6cf5c42f4354a787e9d727c1f2c982d81c579cc0e WatchSource:0}: Error finding container 3d3f758972c1e0c377358ff6cf5c42f4354a787e9d727c1f2c982d81c579cc0e: Status 404 returned error can't find the container with id 3d3f758972c1e0c377358ff6cf5c42f4354a787e9d727c1f2c982d81c579cc0e
Jul 28 18:10:01.580155 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.580129    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:10:01.580227 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.580199    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:10:01.580260 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.580228    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:10:01.834515 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.834478    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq" event={"ID":"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368","Type":"ContainerStarted","Data":"5857deb78f83a27efc9f04427a0f5913d3cd378621aaea6a8ff83f28c0a41a9d"}
Jul 28 18:10:01.834698 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.834535    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq" event={"ID":"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368","Type":"ContainerStarted","Data":"3d3f758972c1e0c377358ff6cf5c42f4354a787e9d727c1f2c982d81c579cc0e"}
Jul 28 18:10:01.857520 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:01.857384    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq" podStartSLOduration=0.857363518 podStartE2EDuration="857.363518ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:10:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:10:01.856948263 +0000 UTC m=+552.933510262" watchObservedRunningTime="2026-07-28 18:10:01.857363518 +0000 UTC m=+552.933925515"
Jul 28 18:10:02.436672 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:02.436641    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:02.441367 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:02.441344    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:02.839187 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:02.839158    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:02.840250 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:02.840230    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:10:14.971361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:14.971325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:16.755452 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:16.755406    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:17.044775 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.044654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:17.149498 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.149458    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:17.247255 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.247176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:17.411572 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.411494    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:10:17.414922 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.414900    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"
Jul 28 18:10:17.418104 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.418081    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 18:10:17.418206 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.418136    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zlqmd\""
Jul 28 18:10:17.423852 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.423827    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:10:17.453707 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.453680    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:10:17.516397 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.516359    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76jhq\" (UniqueName: \"kubernetes.io/projected/b0da8a38-9108-4c77-a3fc-dfc0c41c52c4-kube-api-access-76jhq\") pod \"limitador-limitador-54d6d988d5-6zqbp\" (UID: \"b0da8a38-9108-4c77-a3fc-dfc0c41c52c4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"
Jul 28 18:10:17.516625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.516414    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b0da8a38-9108-4c77-a3fc-dfc0c41c52c4-config-file\") pod \"limitador-limitador-54d6d988d5-6zqbp\" (UID: \"b0da8a38-9108-4c77-a3fc-dfc0c41c52c4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"
Jul 28 18:10:17.617064 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.617026    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76jhq\" (UniqueName: \"kubernetes.io/projected/b0da8a38-9108-4c77-a3fc-dfc0c41c52c4-kube-api-access-76jhq\") pod \"limitador-limitador-54d6d988d5-6zqbp\" (UID: \"b0da8a38-9108-4c77-a3fc-dfc0c41c52c4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"
Jul 28 18:10:17.617244 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.617076    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b0da8a38-9108-4c77-a3fc-dfc0c41c52c4-config-file\") pod \"limitador-limitador-54d6d988d5-6zqbp\" (UID: \"b0da8a38-9108-4c77-a3fc-dfc0c41c52c4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"
Jul 28 18:10:17.617791 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.617771    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b0da8a38-9108-4c77-a3fc-dfc0c41c52c4-config-file\") pod \"limitador-limitador-54d6d988d5-6zqbp\" (UID: \"b0da8a38-9108-4c77-a3fc-dfc0c41c52c4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"
Jul 28 18:10:17.627262 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.627234    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76jhq\" (UniqueName: \"kubernetes.io/projected/b0da8a38-9108-4c77-a3fc-dfc0c41c52c4-kube-api-access-76jhq\") pod \"limitador-limitador-54d6d988d5-6zqbp\" (UID: \"b0da8a38-9108-4c77-a3fc-dfc0c41c52c4\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"
Jul 28 18:10:17.726632 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.726602    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"
Jul 28 18:10:17.870575 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.870540    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:10:17.873652 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:10:17.873621    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0da8a38_9108_4c77_a3fc_dfc0c41c52c4.slice/crio-f27104a6cbd6e422c552f9d8fb981f23dc7e40001995a0730c52305c7fa2f549 WatchSource:0}: Error finding container f27104a6cbd6e422c552f9d8fb981f23dc7e40001995a0730c52305c7fa2f549: Status 404 returned error can't find the container with id f27104a6cbd6e422c552f9d8fb981f23dc7e40001995a0730c52305c7fa2f549
Jul 28 18:10:17.892195 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.892158    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:17.918562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.918515    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp" event={"ID":"b0da8a38-9108-4c77-a3fc-dfc0c41c52c4","Type":"ContainerStarted","Data":"f27104a6cbd6e422c552f9d8fb981f23dc7e40001995a0730c52305c7fa2f549"}
Jul 28 18:10:17.994075 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:17.993972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:20.936217 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:20.936182    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp" event={"ID":"b0da8a38-9108-4c77-a3fc-dfc0c41c52c4","Type":"ContainerStarted","Data":"9315ed60c19e2ecc0c120b1b60dea84b728a0c4a74f370fdf22b519c739319e8"}
Jul 28 18:10:20.936693 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:20.936322    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"
Jul 28 18:10:20.958866 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:20.958807    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp" podStartSLOduration=1.427476279 podStartE2EDuration="3.958792431s" totalImagesPullingTime="2.531316152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:10:20.957273007 +0000 UTC m=+572.033835004" watchObservedRunningTime="2026-07-28 18:10:20.958792431 +0000 UTC m=+572.035354427"
Jul 28 18:10:26.836676 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:26.836637    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:31.941710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:31.941665    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"
Jul 28 18:10:34.423904 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:34.423872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:34.528192 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:34.528127    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:34.909657 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:34.909616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:48.263630 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:48.263598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:48.681438 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:48.681351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:49.325068 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:49.325042    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:10:49.325463 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:49.325043    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:10:55.375139 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:55.375051    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:55.468494 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:55.468444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:55.557267 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:55.557200    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:58.366029 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:58.365994    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:58.470603 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:10:58.470569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:33.901306 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:33.901264    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:35.330743 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:35.330704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:35.349115 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:35.349073    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:11:43.345613 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:43.345576    2583 generic.go:356] "Generic (PLEG): container finished" podID="ea3a4c2a-2d05-4ce3-8a73-5fbcae566368" containerID="5857deb78f83a27efc9f04427a0f5913d3cd378621aaea6a8ff83f28c0a41a9d" exitCode=0
Jul 28 18:11:43.346057 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:43.345648    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq" event={"ID":"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368","Type":"ContainerDied","Data":"5857deb78f83a27efc9f04427a0f5913d3cd378621aaea6a8ff83f28c0a41a9d"}
Jul 28 18:11:43.346103 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:43.346078    2583 scope.go:117] "RemoveContainer" containerID="5857deb78f83a27efc9f04427a0f5913d3cd378621aaea6a8ff83f28c0a41a9d"
Jul 28 18:11:43.346977 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:43.346950    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:11:43.347034 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:43.347015    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:11:43.347235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:43.347046    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:11:44.258679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:44.258625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:44.356765 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:44.356721    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq" event={"ID":"ea3a4c2a-2d05-4ce3-8a73-5fbcae566368","Type":"ContainerStarted","Data":"f9a13cf59b624a80d841b1bfb88ac5f6f794262f5d60ca25cd4708383ba4f962"}
Jul 28 18:11:44.359188 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:44.359156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:44.436857 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:44.436826    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:11:44.437835 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:44.437785    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-jlmbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 28 18:11:44.438045 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:44.438017    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq" podUID="ea3a4c2a-2d05-4ce3-8a73-5fbcae566368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 28 18:11:45.436399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:45.436359    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-jlmbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 28 18:11:45.437021 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:45.436988    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq" podUID="ea3a4c2a-2d05-4ce3-8a73-5fbcae566368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 28 18:11:46.436337 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:46.436295    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-jlmbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 28 18:11:46.436626 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:46.436590    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq" podUID="ea3a4c2a-2d05-4ce3-8a73-5fbcae566368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 28 18:11:47.437159 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:47.437114    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-jlmbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 28 18:11:47.437640 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:47.437181    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq" podUID="ea3a4c2a-2d05-4ce3-8a73-5fbcae566368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 28 18:11:47.840115 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:47.840064    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:11:48.439762 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:48.439732    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:11:48.440774 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:48.440755    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-jlmbq"
Jul 28 18:11:56.953460 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:56.953423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:57.397882 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:11:57.397778    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:12:16.399982 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:16.399936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:17.109421 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:17.109381    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:12:17.547620 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:17.547586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:24.219142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:24.219108    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:25.221183 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:25.221145    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:12:30.788361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.788325    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"]
Jul 28 18:12:30.792039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.792023    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:30.796576 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.796553    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 18:12:30.796693 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.796579    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 18:12:30.796693 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.796553    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-87r8d\""
Jul 28 18:12:30.796693 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.796622    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 18:12:30.803024 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.803005    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"]
Jul 28 18:12:30.827589 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.827556    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:30.917307 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.917273    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:30.917493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.917328    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:30.917493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.917390    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:30.917493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.917437    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4zhg\" (UniqueName: \"kubernetes.io/projected/f31a69aa-9651-4768-88c5-f9e394b3ddb6-kube-api-access-j4zhg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:30.917493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.917480    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:30.917645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:30.917554    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f31a69aa-9651-4768-88c5-f9e394b3ddb6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.018780 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.018742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f31a69aa-9651-4768-88c5-f9e394b3ddb6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.018969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.018829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.018969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.018874    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.018969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.018904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.018969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.018943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4zhg\" (UniqueName: \"kubernetes.io/projected/f31a69aa-9651-4768-88c5-f9e394b3ddb6-kube-api-access-j4zhg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.019177 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.018996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.019346 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.019318    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.019470 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.019358    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.019470 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.019377    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.021156 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.021135    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f31a69aa-9651-4768-88c5-f9e394b3ddb6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.021314 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.021296    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f31a69aa-9651-4768-88c5-f9e394b3ddb6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.028089 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.028065    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4zhg\" (UniqueName: \"kubernetes.io/projected/f31a69aa-9651-4768-88c5-f9e394b3ddb6-kube-api-access-j4zhg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr\" (UID: \"f31a69aa-9651-4768-88c5-f9e394b3ddb6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.102864 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.102757    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:31.237138 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.237097    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"]
Jul 28 18:12:31.238240 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:12:31.238213    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf31a69aa_9651_4768_88c5_f9e394b3ddb6.slice/crio-2019908126c8445b85262ba8413dd85d6e15582d464cef814e7717184355e510 WatchSource:0}: Error finding container 2019908126c8445b85262ba8413dd85d6e15582d464cef814e7717184355e510: Status 404 returned error can't find the container with id 2019908126c8445b85262ba8413dd85d6e15582d464cef814e7717184355e510
Jul 28 18:12:31.239587 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.239568    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:12:31.591162 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.591119    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr" event={"ID":"f31a69aa-9651-4768-88c5-f9e394b3ddb6","Type":"ContainerStarted","Data":"2019908126c8445b85262ba8413dd85d6e15582d464cef814e7717184355e510"}
Jul 28 18:12:31.896842 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:31.896740    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:12:32.597526 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:32.597427    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr" event={"ID":"f31a69aa-9651-4768-88c5-f9e394b3ddb6","Type":"ContainerStarted","Data":"477aeef5b804e0c9ef11931346ecdadd8a1fe5e97e7bcc515978fc4bf74ca39f"}
Jul 28 18:12:32.623146 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:32.623090    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr" podStartSLOduration=1.636797161 podStartE2EDuration="2.623072901s" totalImagesPullingTime="986.27574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:12:32.62187703 +0000 UTC m=+703.698439027" watchObservedRunningTime="2026-07-28 18:12:32.623072901 +0000 UTC m=+703.699634898"
Jul 28 18:12:35.216867 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:35.216835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:37.504956 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:37.504919    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:12:39.573529 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:39.573491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:41.102915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:41.102834    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:41.102915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:41.102900    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:41.115742 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:41.115697    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:41.651186 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:41.651159    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr"
Jul 28 18:12:47.300572 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:47.300530    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:12:50.601612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:50.601580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:50.702056 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:12:50.702020    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:05.213078 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:13:05.213044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:05.314952 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:13:05.314921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:54.556031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:13:54.555977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:54.653989 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:13:54.653959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:55.752009 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:13:55.751974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:55.854587 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:13:55.854546    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:14:06.059439 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:14:06.059397    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:14:06.161844 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:14:06.161774    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:14:24.699557 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:14:24.699473    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:14:34.823283 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:14:34.823246    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:14:44.897010 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:14:44.896974    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:15:08.978150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:15:08.978120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:15:15.439449 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:15:15.439411    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:15:49.359185 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:15:49.359108    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:15:49.359775 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:15:49.359460    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:16:02.299570 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:02.299535    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:03.820981 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:03.820943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:16.067581 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:16.067545    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:25.897768 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:25.897727    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:16:41.904082 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:41.904046    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:16:52.770769 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:52.770733    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:56.367767 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:56.367728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:59.093801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:59.093769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:59.156264 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:59.156213    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:59.237935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:59.237905    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:59.351712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:16:59.351618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:00.269024 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:17:00.268989    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:05.609124 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:17:05.609082    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:05.677081 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:17:05.677044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:22.096009 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:17:22.095927    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:17:32.196128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:17:32.196088    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:17:39.414651 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:17:39.414617    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:17:50.520471 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:17:50.520433    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:18:01.307046 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:18:01.307009    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:18:10.500346 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:18:10.500308    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:18:22.403329 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:18:22.403291    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:18:32.120476 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:18:32.120440    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:18:40.920875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:18:40.920838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:54.601095 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:18:54.601055    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:19:06.696228 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:19:06.696184    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:19:30.296284 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:19:30.296238    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:19:40.610554 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:19:40.610517    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:19:50.801298 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:19:50.801261    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:20:07.119865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:20:07.119828    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:20:13.472536 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:20:13.472428    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:20:16.298400 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:20:16.298362    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:20:26.007192 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:20:26.007157    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:20:38.393493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:20:38.393453    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:20:49.124956 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:20:49.124920    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:20:49.389045 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:20:49.388967    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:20:49.391072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:20:49.391044    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:20:59.198534 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:20:59.198497    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:21:20.621854 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:20.621803    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"]
Jul 28 18:21:20.622334 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:20.622027    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9" podUID="37d9fa15-b0c1-4264-a08d-7e0ebd8861b9" containerName="manager" containerID="cri-o://b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794" gracePeriod=10
Jul 28 18:21:23.970542 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:23.970519    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:21:24.076170 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.076085    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-extensions-socket-volume\") pod \"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9\" (UID: \"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9\") "
Jul 28 18:21:24.076170 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.076157    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk8lx\" (UniqueName: \"kubernetes.io/projected/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-kube-api-access-xk8lx\") pod \"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9\" (UID: \"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9\") "
Jul 28 18:21:24.076463 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.076437    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "37d9fa15-b0c1-4264-a08d-7e0ebd8861b9" (UID: "37d9fa15-b0c1-4264-a08d-7e0ebd8861b9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:21:24.078207 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.078184    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-kube-api-access-xk8lx" (OuterVolumeSpecName: "kube-api-access-xk8lx") pod "37d9fa15-b0c1-4264-a08d-7e0ebd8861b9" (UID: "37d9fa15-b0c1-4264-a08d-7e0ebd8861b9"). InnerVolumeSpecName "kube-api-access-xk8lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:21:24.177627 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.177594    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-extensions-socket-volume\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:21:24.177627 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.177622    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk8lx\" (UniqueName: \"kubernetes.io/projected/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9-kube-api-access-xk8lx\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:21:24.198604 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.198575    2583 generic.go:356] "Generic (PLEG): container finished" podID="37d9fa15-b0c1-4264-a08d-7e0ebd8861b9" containerID="b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794" exitCode=0
Jul 28 18:21:24.198735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.198619    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"
Jul 28 18:21:24.198735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.198659    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9" event={"ID":"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9","Type":"ContainerDied","Data":"b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794"}
Jul 28 18:21:24.198735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.198713    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9" event={"ID":"37d9fa15-b0c1-4264-a08d-7e0ebd8861b9","Type":"ContainerDied","Data":"eb0cd25eb65b5baa19b1afb5dcf175a7b73f9784b098a86c2da0a5d1e396bc0b"}
Jul 28 18:21:24.198870 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.198736    2583 scope.go:117] "RemoveContainer" containerID="b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794"
Jul 28 18:21:24.207860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.207804    2583 scope.go:117] "RemoveContainer" containerID="b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794"
Jul 28 18:21:24.208098 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:21:24.208082    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794\": container with ID starting with b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794 not found: ID does not exist" containerID="b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794"
Jul 28 18:21:24.208150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.208105    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794"} err="failed to get container status \"b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794\": rpc error: code = NotFound desc = could not find container \"b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794\": container with ID starting with b904daa853c49d1d3f40fd0ea00a9d1bbae48705f9131e46e15cbaf875e73794 not found: ID does not exist"
Jul 28 18:21:24.221036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.221012    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"]
Jul 28 18:21:24.225478 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:24.225452    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bbxc9"]
Jul 28 18:21:25.406210 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:21:25.406175    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37d9fa15-b0c1-4264-a08d-7e0ebd8861b9" path="/var/lib/kubelet/pods/37d9fa15-b0c1-4264-a08d-7e0ebd8861b9/volumes"
Jul 28 18:22:21.815044 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:21.815008    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"]
Jul 28 18:22:21.816570 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:21.815405    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37d9fa15-b0c1-4264-a08d-7e0ebd8861b9" containerName="manager"
Jul 28 18:22:21.816570 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:21.815417    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="37d9fa15-b0c1-4264-a08d-7e0ebd8861b9" containerName="manager"
Jul 28 18:22:21.816570 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:21.815491    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="37d9fa15-b0c1-4264-a08d-7e0ebd8861b9" containerName="manager"
Jul 28 18:22:21.817469 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:21.817453    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"
Jul 28 18:22:21.822845 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:21.822809    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mtdpn\""
Jul 28 18:22:21.833147 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:21.833124    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"]
Jul 28 18:22:21.925320 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:21.925288    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2v9j\" (UniqueName: \"kubernetes.io/projected/a409a38d-bfb3-4080-9141-8ca70955c03e-kube-api-access-p2v9j\") pod \"kuadrant-operator-controller-manager-86878c769f-wz2tr\" (UID: \"a409a38d-bfb3-4080-9141-8ca70955c03e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"
Jul 28 18:22:21.925493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:21.925334    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a409a38d-bfb3-4080-9141-8ca70955c03e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wz2tr\" (UID: \"a409a38d-bfb3-4080-9141-8ca70955c03e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"
Jul 28 18:22:22.026832 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:22.026784    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a409a38d-bfb3-4080-9141-8ca70955c03e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wz2tr\" (UID: \"a409a38d-bfb3-4080-9141-8ca70955c03e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"
Jul 28 18:22:22.027029 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:22.026960    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2v9j\" (UniqueName: \"kubernetes.io/projected/a409a38d-bfb3-4080-9141-8ca70955c03e-kube-api-access-p2v9j\") pod \"kuadrant-operator-controller-manager-86878c769f-wz2tr\" (UID: \"a409a38d-bfb3-4080-9141-8ca70955c03e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"
Jul 28 18:22:22.027191 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:22.027165    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a409a38d-bfb3-4080-9141-8ca70955c03e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wz2tr\" (UID: \"a409a38d-bfb3-4080-9141-8ca70955c03e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"
Jul 28 18:22:22.044483 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:22.044451    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2v9j\" (UniqueName: \"kubernetes.io/projected/a409a38d-bfb3-4080-9141-8ca70955c03e-kube-api-access-p2v9j\") pod \"kuadrant-operator-controller-manager-86878c769f-wz2tr\" (UID: \"a409a38d-bfb3-4080-9141-8ca70955c03e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"
Jul 28 18:22:22.128218 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:22.128128    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"
Jul 28 18:22:22.262823 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:22.262793    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"]
Jul 28 18:22:22.266451 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:22:22.266425    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda409a38d_bfb3_4080_9141_8ca70955c03e.slice/crio-992bb87e9e22887118c60da4d31f139cf8311ffed7066f39bae487001fd5500a WatchSource:0}: Error finding container 992bb87e9e22887118c60da4d31f139cf8311ffed7066f39bae487001fd5500a: Status 404 returned error can't find the container with id 992bb87e9e22887118c60da4d31f139cf8311ffed7066f39bae487001fd5500a
Jul 28 18:22:22.487876 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:22.487842    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr" event={"ID":"a409a38d-bfb3-4080-9141-8ca70955c03e","Type":"ContainerStarted","Data":"6c7263ca0915467bf0cfd82112f94acadd5849b1021b6837294062fe7ec44266"}
Jul 28 18:22:22.488050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:22.487896    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr" event={"ID":"a409a38d-bfb3-4080-9141-8ca70955c03e","Type":"ContainerStarted","Data":"992bb87e9e22887118c60da4d31f139cf8311ffed7066f39bae487001fd5500a"}
Jul 28 18:22:22.488050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:22.488029    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"
Jul 28 18:22:22.513937 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:22.513892    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr" podStartSLOduration=1.5138790370000001 podStartE2EDuration="1.513879037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:22:22.512939953 +0000 UTC m=+1293.589501949" watchObservedRunningTime="2026-07-28 18:22:22.513879037 +0000 UTC m=+1293.590441033"
Jul 28 18:22:33.494268 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:33.494233    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wz2tr"
Jul 28 18:22:43.902402 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:43.902355    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:22:54.603113 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:22:54.603074    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:23:09.802112 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:23:09.802072    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:23:21.304285 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:23:21.304196    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:23:26.154198 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:23:26.154166    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:23:31.602086 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:23:31.602058    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:23:39.800519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:23:39.800482    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:23:50.401158 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:23:50.401124    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:24:02.606527 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:24:02.606488    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:24:10.913596 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:24:10.913560    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:24:22.201908 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:24:22.201870    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:24:34.506369 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:24:34.506331    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:24:42.805972 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:24:42.805889    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:24:45.508544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:24:45.508512    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:25:03.796457 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:03.796419    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:25:11.522022 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:11.521980    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:25:21.044132 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.044094    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"]
Jul 28 18:25:21.047474 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.047458    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.052439 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.052421    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-6ed4c9b0-kserve-self-signed-certs\""
Jul 28 18:25:21.086404 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.086367    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"]
Jul 28 18:25:21.091795 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.091771    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-home\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.091925 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.091831    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-dshm\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.091925 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.091859    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-model-cache\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.091999 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.091917    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-tmp-dir\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.091999 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.091961    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmnx6\" (UniqueName: \"kubernetes.io/projected/07f50629-82d4-455b-a3d1-cf305e547427-kube-api-access-fmnx6\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.091999 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.091987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07f50629-82d4-455b-a3d1-cf305e547427-tls-certs\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.113222 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.113170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:25:21.193361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.193328    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-home\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.193552 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.193378    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-dshm\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.193552 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.193395    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-model-cache\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.193552 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.193420    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-tmp-dir\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.193552 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.193455    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmnx6\" (UniqueName: \"kubernetes.io/projected/07f50629-82d4-455b-a3d1-cf305e547427-kube-api-access-fmnx6\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.193552 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.193476    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07f50629-82d4-455b-a3d1-cf305e547427-tls-certs\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.193856 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.193760    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-home\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.193926 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.193886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-model-cache\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.193926 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.193915    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-tmp-dir\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.195687 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.195668    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-dshm\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.195859 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.195843    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07f50629-82d4-455b-a3d1-cf305e547427-tls-certs\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.204443 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.204425    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmnx6\" (UniqueName: \"kubernetes.io/projected/07f50629-82d4-455b-a3d1-cf305e547427-kube-api-access-fmnx6\") pod \"e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") " pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.356884 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.356763    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:21.502557 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:21.502533    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"]
Jul 28 18:25:21.505093 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:25:21.505065    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07f50629_82d4_455b_a3d1_cf305e547427.slice/crio-85f14962687a5370e7e81e5b902e06d2c1bf799348f345de239fc9c735b1938d WatchSource:0}: Error finding container 85f14962687a5370e7e81e5b902e06d2c1bf799348f345de239fc9c735b1938d: Status 404 returned error can't find the container with id 85f14962687a5370e7e81e5b902e06d2c1bf799348f345de239fc9c735b1938d
Jul 28 18:25:22.371327 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:22.371285    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg" event={"ID":"07f50629-82d4-455b-a3d1-cf305e547427","Type":"ContainerStarted","Data":"684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2"}
Jul 28 18:25:22.371716 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:22.371335    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg" event={"ID":"07f50629-82d4-455b-a3d1-cf305e547427","Type":"ContainerStarted","Data":"85f14962687a5370e7e81e5b902e06d2c1bf799348f345de239fc9c735b1938d"}
Jul 28 18:25:22.392978 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:22.392928    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg" podStartSLOduration=1.392911231 podStartE2EDuration="1.392911231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:25:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:25:22.391773846 +0000 UTC m=+1473.468335845" watchObservedRunningTime="2026-07-28 18:25:22.392911231 +0000 UTC m=+1473.469473227"
Jul 28 18:25:24.017844 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:24.017794    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:25:31.357480 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:31.357381    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:31.357480 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:31.357428    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:31.370061 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:31.370034    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:31.426386 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:31.426354    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:25:31.736616 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:31.736580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:25:38.303108 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:38.303072    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:25:47.820153 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:47.818432    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"]
Jul 28 18:25:48.749875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:48.749840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:25:49.422739 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:49.422713    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:25:49.425048 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:49.425029    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:25:49.629141 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:49.629110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:25:51.360748 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:51.360711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:25:53.803889 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:25:53.803855    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:26:02.839686 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:02.839633    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg" podUID="07f50629-82d4-455b-a3d1-cf305e547427" containerName="main" containerID="cri-o://684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2" gracePeriod=45
Jul 28 18:26:03.087014 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.086984    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:26:03.201115 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.201079    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-model-cache\") pod \"07f50629-82d4-455b-a3d1-cf305e547427\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") "
Jul 28 18:26:03.201115 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.201116    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmnx6\" (UniqueName: \"kubernetes.io/projected/07f50629-82d4-455b-a3d1-cf305e547427-kube-api-access-fmnx6\") pod \"07f50629-82d4-455b-a3d1-cf305e547427\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") "
Jul 28 18:26:03.201324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.201170    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07f50629-82d4-455b-a3d1-cf305e547427-tls-certs\") pod \"07f50629-82d4-455b-a3d1-cf305e547427\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") "
Jul 28 18:26:03.201324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.201204    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-home\") pod \"07f50629-82d4-455b-a3d1-cf305e547427\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") "
Jul 28 18:26:03.201324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.201280    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-dshm\") pod \"07f50629-82d4-455b-a3d1-cf305e547427\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") "
Jul 28 18:26:03.201324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.201298    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-tmp-dir\") pod \"07f50629-82d4-455b-a3d1-cf305e547427\" (UID: \"07f50629-82d4-455b-a3d1-cf305e547427\") "
Jul 28 18:26:03.201551 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.201360    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-model-cache" (OuterVolumeSpecName: "model-cache") pod "07f50629-82d4-455b-a3d1-cf305e547427" (UID: "07f50629-82d4-455b-a3d1-cf305e547427"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:26:03.201551 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.201531    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-home" (OuterVolumeSpecName: "home") pod "07f50629-82d4-455b-a3d1-cf305e547427" (UID: "07f50629-82d4-455b-a3d1-cf305e547427"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:26:03.201660 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.201595    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-model-cache\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:26:03.201660 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.201618    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07f50629-82d4-455b-a3d1-cf305e547427" (UID: "07f50629-82d4-455b-a3d1-cf305e547427"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:26:03.203378 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.203354    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f50629-82d4-455b-a3d1-cf305e547427-kube-api-access-fmnx6" (OuterVolumeSpecName: "kube-api-access-fmnx6") pod "07f50629-82d4-455b-a3d1-cf305e547427" (UID: "07f50629-82d4-455b-a3d1-cf305e547427"). InnerVolumeSpecName "kube-api-access-fmnx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:26:03.203715 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.203691    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-dshm" (OuterVolumeSpecName: "dshm") pod "07f50629-82d4-455b-a3d1-cf305e547427" (UID: "07f50629-82d4-455b-a3d1-cf305e547427"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:26:03.203715 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.203696    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07f50629-82d4-455b-a3d1-cf305e547427-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07f50629-82d4-455b-a3d1-cf305e547427" (UID: "07f50629-82d4-455b-a3d1-cf305e547427"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:26:03.303018 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.302972    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-dshm\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:26:03.303018 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.303014    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-tmp-dir\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:26:03.303018 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.303026    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmnx6\" (UniqueName: \"kubernetes.io/projected/07f50629-82d4-455b-a3d1-cf305e547427-kube-api-access-fmnx6\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:26:03.303018 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.303036    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07f50629-82d4-455b-a3d1-cf305e547427-tls-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:26:03.303290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.303044    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07f50629-82d4-455b-a3d1-cf305e547427-home\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:26:03.579096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.579002    2583 generic.go:356] "Generic (PLEG): container finished" podID="07f50629-82d4-455b-a3d1-cf305e547427" containerID="684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2" exitCode=0
Jul 28 18:26:03.579096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.579047    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"
Jul 28 18:26:03.579096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.579083    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg" event={"ID":"07f50629-82d4-455b-a3d1-cf305e547427","Type":"ContainerDied","Data":"684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2"}
Jul 28 18:26:03.579353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.579122    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg" event={"ID":"07f50629-82d4-455b-a3d1-cf305e547427","Type":"ContainerDied","Data":"85f14962687a5370e7e81e5b902e06d2c1bf799348f345de239fc9c735b1938d"}
Jul 28 18:26:03.579353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.579144    2583 scope.go:117] "RemoveContainer" containerID="684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2"
Jul 28 18:26:03.587989 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.587971    2583 scope.go:117] "RemoveContainer" containerID="684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2"
Jul 28 18:26:03.588256 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:26:03.588234    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2\": container with ID starting with 684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2 not found: ID does not exist" containerID="684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2"
Jul 28 18:26:03.588342 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.588268    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2"} err="failed to get container status \"684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2\": rpc error: code = NotFound desc = could not find container \"684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2\": container with ID starting with 684c762ee2e04a2df5a8431ac98fb9de795b895116a908b413c8945d618bc0a2 not found: ID does not exist"
Jul 28 18:26:03.599448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.599422    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"]
Jul 28 18:26:03.606272 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:03.606245    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-6ed4c9b0-kserve-548d5c487-bcrkg"]
Jul 28 18:26:04.601562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:04.601523    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:26:05.405350 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:05.405315    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07f50629-82d4-455b-a3d1-cf305e547427" path="/var/lib/kubelet/pods/07f50629-82d4-455b-a3d1-cf305e547427/volumes"
Jul 28 18:26:10.808347 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:10.808312    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:26:21.466929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:21.466886    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:26:25.601851 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:25.601804    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:26:36.096361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:36.096320    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:26:45.501908 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:45.501876    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:26:54.714134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:26:54.714096    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:27:05.907152 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:27:05.907112    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:27:15.502072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:27:15.502036    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:27:35.890876 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:27:35.890842    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:27:46.798165 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:27:46.798073    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:27:56.396611 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:27:56.396572    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:28:09.091908 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:28:09.091870    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:28:18.093522 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:28:18.093485    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:28:28.900898 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:28:28.900862    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:28:42.297666 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:28:42.297631    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:28:51.497957 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:28:51.497921    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:28:54.264930 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:28:54.264899    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:04.694479 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:04.694444    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:29:14.191690 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:14.191608    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:29:16.232672 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:16.232639    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:19.510152 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:19.510115    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:19.610799 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:19.610772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:20.021556 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:20.021522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:24.288829 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:24.288788    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:29:25.909157 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:25.909124    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:26.013666 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.013634    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:26.174233 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.174152    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:26.485882 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.485848    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"]
Jul 28 18:29:26.486253 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.486240    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f50629-82d4-455b-a3d1-cf305e547427" containerName="main"
Jul 28 18:29:26.486298 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.486254    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f50629-82d4-455b-a3d1-cf305e547427" containerName="main"
Jul 28 18:29:26.486343 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.486338    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f50629-82d4-455b-a3d1-cf305e547427" containerName="main"
Jul 28 18:29:26.489432 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.489415    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.492484 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.492462    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-314f4856-openshift-default-dockercfg-645m6\""
Jul 28 18:29:26.501755 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.501732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:26.511939 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.511917    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"]
Jul 28 18:29:26.612918 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.612889    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-token\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.613087 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.612926    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-envoy\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.613087 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.613022    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-certs\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.613087 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.613056    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-socket\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.613087 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.613086    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-podinfo\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.613283 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.613146    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh5hs\" (UniqueName: \"kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-kube-api-access-xh5hs\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.613283 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.613228    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-credential-socket\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.613283 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.613260    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istiod-ca-cert\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.613395 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.613306    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-data\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.623673 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.623645    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:26.714051 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714021    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-data\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714072    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-token\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-envoy\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714134    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-certs\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714156    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-socket\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714188    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-podinfo\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh5hs\" (UniqueName: \"kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-kube-api-access-xh5hs\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714550 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714239    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-credential-socket\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714550 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-data\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714550 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istiod-ca-cert\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714550 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714537    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-credential-socket\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714771 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714564    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-certs\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.714867 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.714783    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-socket\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.715098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.715076    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istiod-ca-cert\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.716408 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.716390    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-envoy\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.716581 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.716565    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-podinfo\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.722429 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.722406    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-token\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.722611 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.722589    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh5hs\" (UniqueName: \"kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-kube-api-access-xh5hs\") pod \"e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") " pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.799503 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.799421    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:26.946198 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.946170    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"]
Jul 28 18:29:26.948437 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:29:26.948410    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5059990_395a_4a73_a5a5_ebefc0cf76ba.slice/crio-455b64afbf0ec1722356dcaa773d68de4851c1ae9b2bbb64ec05895df2fd2423 WatchSource:0}: Error finding container 455b64afbf0ec1722356dcaa773d68de4851c1ae9b2bbb64ec05895df2fd2423: Status 404 returned error can't find the container with id 455b64afbf0ec1722356dcaa773d68de4851c1ae9b2bbb64ec05895df2fd2423
Jul 28 18:29:26.950612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.950583    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:29:26.950691 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.950645    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:29:26.950691 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:26.950672    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:29:27.554498 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:27.554449    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" event={"ID":"b5059990-395a-4a73-a5a5-ebefc0cf76ba","Type":"ContainerStarted","Data":"a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7"}
Jul 28 18:29:27.554687 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:27.554518    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" event={"ID":"b5059990-395a-4a73-a5a5-ebefc0cf76ba","Type":"ContainerStarted","Data":"455b64afbf0ec1722356dcaa773d68de4851c1ae9b2bbb64ec05895df2fd2423"}
Jul 28 18:29:27.583304 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:27.583251    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" podStartSLOduration=1.583235172 podStartE2EDuration="1.583235172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:29:27.580695458 +0000 UTC m=+1718.657257455" watchObservedRunningTime="2026-07-28 18:29:27.583235172 +0000 UTC m=+1718.659797147"
Jul 28 18:29:27.799865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:27.799796    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:28.805225 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:28.805190    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:28.933144 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:28.933094    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:29.014328 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:29.014288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:29.182207 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:29.182115    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:29.424991 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:29.424960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:29.566851 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:29.566077    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:29.570599 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:29.570573    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:29.721330 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:29.721297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:29.817041 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:29.816957    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:35.095961 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:35.095927    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:29:43.638480 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:43.638443    2583 generic.go:356] "Generic (PLEG): container finished" podID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerID="a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7" exitCode=0
Jul 28 18:29:43.639023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:43.638488    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" event={"ID":"b5059990-395a-4a73-a5a5-ebefc0cf76ba","Type":"ContainerDied","Data":"a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7"}
Jul 28 18:29:43.639023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:43.638897    2583 scope.go:117] "RemoveContainer" containerID="a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7"
Jul 28 18:29:43.639911 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:43.639883    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:29:43.639994 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:43.639951    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:29:43.639994 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:43.639982    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:29:44.566570 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:44.566535    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:44.649057 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:44.649020    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" event={"ID":"b5059990-395a-4a73-a5a5-ebefc0cf76ba","Type":"ContainerStarted","Data":"26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8"}
Jul 28 18:29:44.800600 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:44.800570    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:44.801799 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:44.801773    2583 patch_prober.go:28] interesting pod/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 18:29:44.801930 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:44.801843    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 18:29:45.800872 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:45.800809    2583 patch_prober.go:28] interesting pod/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 18:29:45.801276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:45.800903    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 18:29:46.800207 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:46.800169    2583 patch_prober.go:28] interesting pod/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 18:29:46.800418 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:46.800230    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 18:29:47.800548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:47.800514    2583 patch_prober.go:28] interesting pod/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 28 18:29:47.800981 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:47.800566    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 28 18:29:48.804085 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:48.804057    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:48.804588 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:48.804441    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:48.805152 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:48.805130    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:29:59.605234 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:59.605195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:59.664761 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:59.664718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:59.729627 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:59.729590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:59.790031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:59.789999    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:59.910000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:59.909905    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"]
Jul 28 18:29:59.910240 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:59.910213    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy" containerID="cri-o://26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8" gracePeriod=30
Jul 28 18:29:59.926219 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:29:59.926187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:00.153947 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:00.153920    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:00.409439 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:00.409396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:00.508801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:00.508759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:00.639883 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:00.639852    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:00.955333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:00.955300    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"]
Jul 28 18:30:00.958788 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:00.958768    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:00.961725 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:00.961710    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ba40d93a-gw-openshift-default-dockercfg-fxd67\""
Jul 28 18:30:00.972471 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:00.972449    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"]
Jul 28 18:30:00.977557 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:00.977538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:01.049099 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.049068    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-certs\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.049099 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.049106    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-socket\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.049341 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.049151    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-credential-socket\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.049341 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.049173    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-token\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.049341 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.049264    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-envoy\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.049341 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.049282    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28246\" (UniqueName: \"kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-kube-api-access-28246\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.049341 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.049307    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-data\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.049341 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.049340    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-podinfo\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.049548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.049364    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebfd1db6-0397-4b99-90cc-db3e394f1613-istiod-ca-cert\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.150488 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.150456    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-envoy\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.150488 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.150489    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28246\" (UniqueName: \"kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-kube-api-access-28246\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.150704 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.150513    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-data\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.150704 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.150544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-podinfo\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.150704 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.150665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebfd1db6-0397-4b99-90cc-db3e394f1613-istiod-ca-cert\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.150873 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.150737    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-certs\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.150873 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.150766    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-socket\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.150873 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.150836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-credential-socket\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.150873 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.150865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-token\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.151096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.150969    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-data\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.151151 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.151136    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-socket\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.151208 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.151176    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-certs\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.151298 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.151267    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-credential-socket\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.151537 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.151510    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebfd1db6-0397-4b99-90cc-db3e394f1613-istiod-ca-cert\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.152883 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.152862    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-envoy\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.153150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.153132    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-podinfo\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.162654 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.162630    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28246\" (UniqueName: \"kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-kube-api-access-28246\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.163020 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.162998    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-token\") pod \"e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") " pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.269236 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.269140    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:01.413346 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.413324    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"]
Jul 28 18:30:01.415580 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:30:01.415548    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfd1db6_0397_4b99_90cc_db3e394f1613.slice/crio-a1f1f5dca62f15036e7410a2dcf442b1334f0330eb52ba6ab84c433ed08f2ccf WatchSource:0}: Error finding container a1f1f5dca62f15036e7410a2dcf442b1334f0330eb52ba6ab84c433ed08f2ccf: Status 404 returned error can't find the container with id a1f1f5dca62f15036e7410a2dcf442b1334f0330eb52ba6ab84c433ed08f2ccf
Jul 28 18:30:01.417560 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.417532    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:30:01.417637 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.417594    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:30:01.417637 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.417624    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:30:01.473529 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.473490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:01.744571 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.744533    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" event={"ID":"ebfd1db6-0397-4b99-90cc-db3e394f1613","Type":"ContainerStarted","Data":"8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534"}
Jul 28 18:30:01.744997 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.744593    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" event={"ID":"ebfd1db6-0397-4b99-90cc-db3e394f1613","Type":"ContainerStarted","Data":"a1f1f5dca62f15036e7410a2dcf442b1334f0330eb52ba6ab84c433ed08f2ccf"}
Jul 28 18:30:01.767298 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:01.767237    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" podStartSLOduration=1.767223456 podStartE2EDuration="1.767223456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:30:01.764690375 +0000 UTC m=+1752.841252372" watchObservedRunningTime="2026-07-28 18:30:01.767223456 +0000 UTC m=+1752.843785456"
Jul 28 18:30:02.269492 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:02.269453    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:02.274649 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:02.274626    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:02.750909 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:02.750882    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:02.751984 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:02.751959    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:03.249827 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:03.249795    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:03.473304 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:03.473268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:03.655411 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:03.655325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:03.798842 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:03.798794    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:30:03.807397 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:03.807365    2583 patch_prober.go:28] interesting pod/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 18:30:03.807566 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:03.807420    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 18:30:03.928287 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:03.928208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:04.154363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:04.154330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:05.170869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.170778    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:30:05.296098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296058    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-credential-socket\") pod \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") "
Jul 28 18:30:05.296098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296096    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istiod-ca-cert\") pod \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") "
Jul 28 18:30:05.296356 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296158    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-token\") pod \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") "
Jul 28 18:30:05.296356 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296194    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-data\") pod \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") "
Jul 28 18:30:05.296465 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296350    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh5hs\" (UniqueName: \"kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-kube-api-access-xh5hs\") pod \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") "
Jul 28 18:30:05.296465 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296432    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-certs\") pod \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") "
Jul 28 18:30:05.296465 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296360    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b5059990-395a-4a73-a5a5-ebefc0cf76ba" (UID: "b5059990-395a-4a73-a5a5-ebefc0cf76ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:30:05.296674 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296471    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-envoy\") pod \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") "
Jul 28 18:30:05.296674 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296498    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-podinfo\") pod \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") "
Jul 28 18:30:05.296674 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296517    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-socket\") pod \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\" (UID: \"b5059990-395a-4a73-a5a5-ebefc0cf76ba\") "
Jul 28 18:30:05.296674 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296663    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b5059990-395a-4a73-a5a5-ebefc0cf76ba" (UID: "b5059990-395a-4a73-a5a5-ebefc0cf76ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:30:05.297017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296672    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b5059990-395a-4a73-a5a5-ebefc0cf76ba" (UID: "b5059990-395a-4a73-a5a5-ebefc0cf76ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:30:05.297017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296905    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b5059990-395a-4a73-a5a5-ebefc0cf76ba" (UID: "b5059990-395a-4a73-a5a5-ebefc0cf76ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:30:05.297017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.296982    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:05.297017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.297005    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:05.297227 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.297020    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:05.297227 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.297068    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "b5059990-395a-4a73-a5a5-ebefc0cf76ba" (UID: "b5059990-395a-4a73-a5a5-ebefc0cf76ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:30:05.298593 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.298568    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b5059990-395a-4a73-a5a5-ebefc0cf76ba" (UID: "b5059990-395a-4a73-a5a5-ebefc0cf76ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:30:05.298705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.298658    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b5059990-395a-4a73-a5a5-ebefc0cf76ba" (UID: "b5059990-395a-4a73-a5a5-ebefc0cf76ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:30:05.298705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.298672    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "b5059990-395a-4a73-a5a5-ebefc0cf76ba" (UID: "b5059990-395a-4a73-a5a5-ebefc0cf76ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:30:05.298705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.298695    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-kube-api-access-xh5hs" (OuterVolumeSpecName: "kube-api-access-xh5hs") pod "b5059990-395a-4a73-a5a5-ebefc0cf76ba" (UID: "b5059990-395a-4a73-a5a5-ebefc0cf76ba"). InnerVolumeSpecName "kube-api-access-xh5hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:30:05.398419 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.398371    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh5hs\" (UniqueName: \"kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-kube-api-access-xh5hs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:05.398419 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.398411    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:05.398419 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.398425    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:05.398678 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.398437    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:05.398678 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.398449    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:05.398678 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.398460    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5059990-395a-4a73-a5a5-ebefc0cf76ba-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:05.767124 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.767083    2583 generic.go:356] "Generic (PLEG): container finished" podID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerID="26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8" exitCode=0
Jul 28 18:30:05.767281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.767134    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"
Jul 28 18:30:05.767281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.767165    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" event={"ID":"b5059990-395a-4a73-a5a5-ebefc0cf76ba","Type":"ContainerDied","Data":"26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8"}
Jul 28 18:30:05.767281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.767215    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx" event={"ID":"b5059990-395a-4a73-a5a5-ebefc0cf76ba","Type":"ContainerDied","Data":"455b64afbf0ec1722356dcaa773d68de4851c1ae9b2bbb64ec05895df2fd2423"}
Jul 28 18:30:05.767281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.767237    2583 scope.go:117] "RemoveContainer" containerID="26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8"
Jul 28 18:30:05.778334 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.777685    2583 scope.go:117] "RemoveContainer" containerID="a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7"
Jul 28 18:30:05.787191 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.787167    2583 scope.go:117] "RemoveContainer" containerID="26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8"
Jul 28 18:30:05.787469 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:30:05.787449    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8\": container with ID starting with 26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8 not found: ID does not exist" containerID="26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8"
Jul 28 18:30:05.787521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.787479    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8"} err="failed to get container status \"26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8\": rpc error: code = NotFound desc = could not find container \"26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8\": container with ID starting with 26097bd194148c122939f1e403533d61247df0b8ec8c3d128766436495847da8 not found: ID does not exist"
Jul 28 18:30:05.787521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.787497    2583 scope.go:117] "RemoveContainer" containerID="a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7"
Jul 28 18:30:05.787761 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:30:05.787740    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7\": container with ID starting with a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7 not found: ID does not exist" containerID="a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7"
Jul 28 18:30:05.787845 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.787770    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7"} err="failed to get container status \"a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7\": rpc error: code = NotFound desc = could not find container \"a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7\": container with ID starting with a7eefa7006e6af84bac36dca3a9906d7b150c1dbd5268aaf91a0782cb9b1b0a7 not found: ID does not exist"
Jul 28 18:30:05.789313 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.789294    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"]
Jul 28 18:30:05.792532 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:05.792510    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-314f4856-openshift-default-8474dc5fc8-k72lx"]
Jul 28 18:30:07.406198 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:07.406164    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" path="/var/lib/kubelet/pods/b5059990-395a-4a73-a5a5-ebefc0cf76ba/volumes"
Jul 28 18:30:15.091363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:15.091319    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:30:17.752221 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:17.752180    2583 patch_prober.go:28] interesting pod/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 28 18:30:17.752667 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:17.752240    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 28 18:30:17.830389 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:17.830355    2583 generic.go:356] "Generic (PLEG): container finished" podID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerID="8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534" exitCode=0
Jul 28 18:30:17.830562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:17.830438    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" event={"ID":"ebfd1db6-0397-4b99-90cc-db3e394f1613","Type":"ContainerDied","Data":"8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534"}
Jul 28 18:30:17.830873 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:17.830860    2583 scope.go:117] "RemoveContainer" containerID="8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534"
Jul 28 18:30:17.831953 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:17.831921    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:30:17.832104 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:17.831991    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:30:17.832104 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:17.832024    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:30:18.839152 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:18.839116    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" event={"ID":"ebfd1db6-0397-4b99-90cc-db3e394f1613","Type":"ContainerStarted","Data":"058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f"}
Jul 28 18:30:19.269258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:19.269225    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:19.269527 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:19.269507    2583 patch_prober.go:28] interesting pod/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 28 18:30:19.269578 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:19.269551    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 28 18:30:20.270049 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:20.270016    2583 patch_prober.go:28] interesting pod/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 28 18:30:20.270412 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:20.270068    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 28 18:30:21.270379 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:21.270342    2583 patch_prober.go:28] interesting pod/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 28 18:30:21.270785 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:21.270394    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 28 18:30:22.270389 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:22.270349    2583 patch_prober.go:28] interesting pod/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 28 18:30:22.270768 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:22.270405    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 28 18:30:23.269693 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:23.269656    2583 patch_prober.go:28] interesting pod/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 28 18:30:23.269890 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:23.269715    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 28 18:30:24.273844 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:24.273786    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:24.274291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:24.274272    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:24.274880 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:24.274865    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:44.770041 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:44.769810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:44.870561 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:44.870520    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:44.925560 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:44.925523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:45.025550 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.025460    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"]
Jul 28 18:30:45.025706 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.025667    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy" containerID="cri-o://058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f" gracePeriod=30
Jul 28 18:30:45.047142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.047115    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:45.288805 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.288724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:45.537773 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.537742    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:45.829464 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.829433    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"]
Jul 28 18:30:45.829869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.829804    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy"
Jul 28 18:30:45.829869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.829829    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy"
Jul 28 18:30:45.829869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.829841    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy"
Jul 28 18:30:45.829869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.829846    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy"
Jul 28 18:30:45.830044 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.829922    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy"
Jul 28 18:30:45.830044 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.829932    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5059990-395a-4a73-a5a5-ebefc0cf76ba" containerName="istio-proxy"
Jul 28 18:30:45.833222 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.833205    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.836005 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.835981    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b13f7e6e-openshift-default-dockercfg-vzt9w\""
Jul 28 18:30:45.844142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.844116    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"]
Jul 28 18:30:45.848225 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.848174    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:45.898143 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.898105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-envoy\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.898324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.898147    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-podinfo\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.898324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.898236    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-data\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.898324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.898282    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-token\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.898324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.898312    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcgpf\" (UniqueName: \"kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-kube-api-access-zcgpf\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.898544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.898387    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-socket\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.898544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.898414    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de93880d-134d-45e8-9c07-d6ac2a789adb-istiod-ca-cert\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.898544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.898461    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-certs\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.898544 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.898490    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-credential-socket\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.999622 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.999591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-data\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.999801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.999630    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-token\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.999801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.999655    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcgpf\" (UniqueName: \"kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-kube-api-access-zcgpf\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.999801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.999692    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-socket\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.999801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.999713    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de93880d-134d-45e8-9c07-d6ac2a789adb-istiod-ca-cert\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:45.999801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.999754    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-certs\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.000150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.999803    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-credential-socket\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.000150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.999923    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-envoy\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.000150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:45.999965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-podinfo\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.000150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.000035    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-data\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.000347 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.000148    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-certs\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.000347 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.000247    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-credential-socket\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.000347 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.000272    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-socket\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.000493 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.000474    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de93880d-134d-45e8-9c07-d6ac2a789adb-istiod-ca-cert\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.002030 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.002011    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-envoy\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.002275 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.002258    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-podinfo\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.008644 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.008617    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-token\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.008761 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.008708    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcgpf\" (UniqueName: \"kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-kube-api-access-zcgpf\") pod \"e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") " pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.106259 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.106174    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:46.144769 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.144738    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:46.283221 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.283195    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"]
Jul 28 18:30:46.285167 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:30:46.285139    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde93880d_134d_45e8_9c07_d6ac2a789adb.slice/crio-900e97feaa27dfb953777dca1fd2ebf5743f1316c2179ba37189dd2234b55d9a WatchSource:0}: Error finding container 900e97feaa27dfb953777dca1fd2ebf5743f1316c2179ba37189dd2234b55d9a: Status 404 returned error can't find the container with id 900e97feaa27dfb953777dca1fd2ebf5743f1316c2179ba37189dd2234b55d9a
Jul 28 18:30:46.287236 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.287205    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:30:46.287340 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.287280    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:30:46.287340 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.287307    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:30:46.982737 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.982701    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" event={"ID":"de93880d-134d-45e8-9c07-d6ac2a789adb","Type":"ContainerStarted","Data":"5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7"}
Jul 28 18:30:46.983131 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:46.982753    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" event={"ID":"de93880d-134d-45e8-9c07-d6ac2a789adb","Type":"ContainerStarted","Data":"900e97feaa27dfb953777dca1fd2ebf5743f1316c2179ba37189dd2234b55d9a"}
Jul 28 18:30:47.006399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:47.006338    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" podStartSLOduration=2.006320848 podStartE2EDuration="2.006320848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:30:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:30:47.004266582 +0000 UTC m=+1798.080828593" watchObservedRunningTime="2026-07-28 18:30:47.006320848 +0000 UTC m=+1798.082882845"
Jul 28 18:30:47.145486 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:47.145452    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:47.151476 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:47.151447    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:47.987941 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:47.987912    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:47.988860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:47.988840    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:30:48.335382 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:48.335293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:48.415463 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:48.415424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:48.493487 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:48.493440    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:48.699231 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:48.699198    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:30:48.791642 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:48.791578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:48.925163 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:48.925131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:49.025443 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:49.025355    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:49.455697 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:49.455673    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:30:49.457126 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:49.457104    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:30:50.277489 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.277464    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:50.445974 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.445948    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-token\") pod \"ebfd1db6-0397-4b99-90cc-db3e394f1613\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") "
Jul 28 18:30:50.446135 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.445998    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-certs\") pod \"ebfd1db6-0397-4b99-90cc-db3e394f1613\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") "
Jul 28 18:30:50.446135 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446049    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28246\" (UniqueName: \"kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-kube-api-access-28246\") pod \"ebfd1db6-0397-4b99-90cc-db3e394f1613\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") "
Jul 28 18:30:50.446135 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446086    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebfd1db6-0397-4b99-90cc-db3e394f1613-istiod-ca-cert\") pod \"ebfd1db6-0397-4b99-90cc-db3e394f1613\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") "
Jul 28 18:30:50.446135 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446127    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-socket\") pod \"ebfd1db6-0397-4b99-90cc-db3e394f1613\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") "
Jul 28 18:30:50.446353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446155    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-envoy\") pod \"ebfd1db6-0397-4b99-90cc-db3e394f1613\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") "
Jul 28 18:30:50.446353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446194    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-data\") pod \"ebfd1db6-0397-4b99-90cc-db3e394f1613\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") "
Jul 28 18:30:50.446353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446216    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-credential-socket\") pod \"ebfd1db6-0397-4b99-90cc-db3e394f1613\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") "
Jul 28 18:30:50.446353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446247    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-podinfo\") pod \"ebfd1db6-0397-4b99-90cc-db3e394f1613\" (UID: \"ebfd1db6-0397-4b99-90cc-db3e394f1613\") "
Jul 28 18:30:50.446353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446341    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ebfd1db6-0397-4b99-90cc-db3e394f1613" (UID: "ebfd1db6-0397-4b99-90cc-db3e394f1613"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:30:50.446614 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446547    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ebfd1db6-0397-4b99-90cc-db3e394f1613" (UID: "ebfd1db6-0397-4b99-90cc-db3e394f1613"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:30:50.446614 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446591    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebfd1db6-0397-4b99-90cc-db3e394f1613-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ebfd1db6-0397-4b99-90cc-db3e394f1613" (UID: "ebfd1db6-0397-4b99-90cc-db3e394f1613"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:30:50.446710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.446608    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:50.447248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.447221    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-data" (OuterVolumeSpecName: "istio-data") pod "ebfd1db6-0397-4b99-90cc-db3e394f1613" (UID: "ebfd1db6-0397-4b99-90cc-db3e394f1613"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:30:50.447317 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.447290    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ebfd1db6-0397-4b99-90cc-db3e394f1613" (UID: "ebfd1db6-0397-4b99-90cc-db3e394f1613"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:30:50.448162 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.448139    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-token" (OuterVolumeSpecName: "istio-token") pod "ebfd1db6-0397-4b99-90cc-db3e394f1613" (UID: "ebfd1db6-0397-4b99-90cc-db3e394f1613"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:30:50.448356 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.448338    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ebfd1db6-0397-4b99-90cc-db3e394f1613" (UID: "ebfd1db6-0397-4b99-90cc-db3e394f1613"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:30:50.448573 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.448554    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ebfd1db6-0397-4b99-90cc-db3e394f1613" (UID: "ebfd1db6-0397-4b99-90cc-db3e394f1613"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:30:50.448624 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.448556    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-kube-api-access-28246" (OuterVolumeSpecName: "kube-api-access-28246") pod "ebfd1db6-0397-4b99-90cc-db3e394f1613" (UID: "ebfd1db6-0397-4b99-90cc-db3e394f1613"). InnerVolumeSpecName "kube-api-access-28246". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:30:50.548165 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.548121    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:50.548165 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.548163    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28246\" (UniqueName: \"kubernetes.io/projected/ebfd1db6-0397-4b99-90cc-db3e394f1613-kube-api-access-28246\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:50.548165 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.548173    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebfd1db6-0397-4b99-90cc-db3e394f1613-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:50.548165 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.548182    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:50.548439 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.548190    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:50.548439 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.548198    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:50.548439 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.548206    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebfd1db6-0397-4b99-90cc-db3e394f1613-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:50.548439 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:50.548214    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebfd1db6-0397-4b99-90cc-db3e394f1613-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:30:51.004867 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.004808    2583 generic.go:356] "Generic (PLEG): container finished" podID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerID="058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f" exitCode=0
Jul 28 18:30:51.005039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.004883    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"
Jul 28 18:30:51.005039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.004903    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" event={"ID":"ebfd1db6-0397-4b99-90cc-db3e394f1613","Type":"ContainerDied","Data":"058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f"}
Jul 28 18:30:51.005039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.004946    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb" event={"ID":"ebfd1db6-0397-4b99-90cc-db3e394f1613","Type":"ContainerDied","Data":"a1f1f5dca62f15036e7410a2dcf442b1334f0330eb52ba6ab84c433ed08f2ccf"}
Jul 28 18:30:51.005039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.004968    2583 scope.go:117] "RemoveContainer" containerID="058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f"
Jul 28 18:30:51.014675 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.014655    2583 scope.go:117] "RemoveContainer" containerID="8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534"
Jul 28 18:30:51.022636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.022620    2583 scope.go:117] "RemoveContainer" containerID="058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f"
Jul 28 18:30:51.022866 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:30:51.022848    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f\": container with ID starting with 058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f not found: ID does not exist" containerID="058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f"
Jul 28 18:30:51.022922 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.022877    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f"} err="failed to get container status \"058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f\": rpc error: code = NotFound desc = could not find container \"058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f\": container with ID starting with 058f8a074a78ae9e77b4e847663ba16725d62f58445cc2e6dbf6b78e6ae5e44f not found: ID does not exist"
Jul 28 18:30:51.022922 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.022895    2583 scope.go:117] "RemoveContainer" containerID="8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534"
Jul 28 18:30:51.023142 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:30:51.023125    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534\": container with ID starting with 8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534 not found: ID does not exist" containerID="8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534"
Jul 28 18:30:51.023198 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.023147    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534"} err="failed to get container status \"8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534\": rpc error: code = NotFound desc = could not find container \"8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534\": container with ID starting with 8f0ef36284d14af5c8de1649c6a022d1a2e949b0561a8f41e915e62d20c79534 not found: ID does not exist"
Jul 28 18:30:51.027888 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.027863    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"]
Jul 28 18:30:51.034872 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.034848    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ba40d93a-gw-openshift-default-fcbdf6d5b-wnxkb"]
Jul 28 18:30:51.406207 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:30:51.406125    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" path="/var/lib/kubelet/pods/ebfd1db6-0397-4b99-90cc-db3e394f1613/volumes"
Jul 28 18:31:00.388111 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:00.388073    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:31:02.989106 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:02.989072    2583 patch_prober.go:28] interesting pod/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 28 18:31:02.989530 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:02.989126    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 28 18:31:03.076172 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:03.076138    2583 generic.go:356] "Generic (PLEG): container finished" podID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerID="5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7" exitCode=0
Jul 28 18:31:03.076352 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:03.076221    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" event={"ID":"de93880d-134d-45e8-9c07-d6ac2a789adb","Type":"ContainerDied","Data":"5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7"}
Jul 28 18:31:03.076675 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:03.076661    2583 scope.go:117] "RemoveContainer" containerID="5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7"
Jul 28 18:31:03.077857 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:03.077800    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:31:03.077930 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:03.077913    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:31:03.077972 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:03.077959    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:31:04.082537 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:04.082499    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" event={"ID":"de93880d-134d-45e8-9c07-d6ac2a789adb","Type":"ContainerStarted","Data":"716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327"}
Jul 28 18:31:04.145270 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:04.145222    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:31:04.145527 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:04.145496    2583 patch_prober.go:28] interesting pod/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 28 18:31:04.145648 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:04.145572    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 28 18:31:05.145954 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:05.145913    2583 patch_prober.go:28] interesting pod/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 28 18:31:05.146436 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:05.145990    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 28 18:31:06.146080 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:06.146036    2583 patch_prober.go:28] interesting pod/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 28 18:31:06.146568 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:06.146119    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 28 18:31:07.145602 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:07.145556    2583 patch_prober.go:28] interesting pod/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 28 18:31:07.146066 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:07.146033    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 28 18:31:08.149422 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:08.149390    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:31:08.149841 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:08.149655    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:31:08.150354 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:08.150334    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:31:19.067435 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:19.067401    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:19.138885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:19.138846    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:19.207306 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:19.207272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:19.308096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:19.308065    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:20.367347 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:20.367315    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"]
Jul 28 18:31:20.367774 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:20.367540    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy" containerID="cri-o://716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327" gracePeriod=30
Jul 28 18:31:20.382263 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:20.382238    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:20.600505 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:20.600470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:20.824275 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:20.824247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:21.098381 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.098292    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"]
Jul 28 18:31:21.098753 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.098735    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy"
Jul 28 18:31:21.098858 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.098755    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy"
Jul 28 18:31:21.098858 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.098770    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy"
Jul 28 18:31:21.098858 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.098777    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy"
Jul 28 18:31:21.099019 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.098916    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy"
Jul 28 18:31:21.102179 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.102158    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.104884 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.104864    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-12084f6d-openshift-default-dockercfg-xzxlt\""
Jul 28 18:31:21.114269 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.114247    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"]
Jul 28 18:31:21.120557 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.120536    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:21.249001 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.248963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-envoy\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.249169 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.249013    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6852c94b-9494-4b6b-955e-8a8681342d79-istio-podinfo\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.249169 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.249057    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-data\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.249169 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.249086    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-socket\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.249169 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.249130    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wgvl\" (UniqueName: \"kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-kube-api-access-6wgvl\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.249169 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.249152    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-certs\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.249169 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.249170    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-credential-socket\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.249375 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.249191    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-istio-token\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.249375 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.249209    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6852c94b-9494-4b6b-955e-8a8681342d79-istiod-ca-cert\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350156 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350071    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-istio-token\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350156 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6852c94b-9494-4b6b-955e-8a8681342d79-istiod-ca-cert\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350162    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-envoy\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350196    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6852c94b-9494-4b6b-955e-8a8681342d79-istio-podinfo\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350269    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-data\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350369    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-socket\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350435    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wgvl\" (UniqueName: \"kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-kube-api-access-6wgvl\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-certs\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350501    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-credential-socket\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350737 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350640    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-data\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350918 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-certs\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.350918 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.350913    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-credential-socket\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.351125 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.351023    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-socket\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.351125 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.351030    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6852c94b-9494-4b6b-955e-8a8681342d79-istiod-ca-cert\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.352619 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.352599    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-envoy\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.352850 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.352832    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6852c94b-9494-4b6b-955e-8a8681342d79-istio-podinfo\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.358038 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.358020    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wgvl\" (UniqueName: \"kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-kube-api-access-6wgvl\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.358128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.358091    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-istio-token\") pod \"e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") " pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.413384 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.413351    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:21.456887 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.456853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:21.562337 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.562304    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"]
Jul 28 18:31:21.563688 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:31:21.563663    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6852c94b_9494_4b6b_955e_8a8681342d79.slice/crio-befbd58be422658231fadf03b9fb265bcfd9ce82171e9e359294cac4911b14e8 WatchSource:0}: Error finding container befbd58be422658231fadf03b9fb265bcfd9ce82171e9e359294cac4911b14e8: Status 404 returned error can't find the container with id befbd58be422658231fadf03b9fb265bcfd9ce82171e9e359294cac4911b14e8
Jul 28 18:31:21.566019 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.565981    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:31:21.566120 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.566062    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:31:21.566120 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:21.566093    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:31:22.181175 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:22.181141    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" event={"ID":"6852c94b-9494-4b6b-955e-8a8681342d79","Type":"ContainerStarted","Data":"6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440"}
Jul 28 18:31:22.181353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:22.181194    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" event={"ID":"6852c94b-9494-4b6b-955e-8a8681342d79","Type":"ContainerStarted","Data":"befbd58be422658231fadf03b9fb265bcfd9ce82171e9e359294cac4911b14e8"}
Jul 28 18:31:22.205274 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:22.205219    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" podStartSLOduration=1.205204208 podStartE2EDuration="1.205204208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:31:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:31:22.203277364 +0000 UTC m=+1833.279839374" watchObservedRunningTime="2026-07-28 18:31:22.205204208 +0000 UTC m=+1833.281766204"
Jul 28 18:31:22.414026 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:22.413990    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:22.418983 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:22.418959    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:22.988469 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:22.988431    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:31:23.151915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:23.151870    2583 patch_prober.go:28] interesting pod/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 18:31:23.152078 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:23.151936    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 18:31:23.186024 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:23.185989    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:23.186825 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:23.186794    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:23.574653 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:23.574617    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:23.709329 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:23.709293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:23.791329 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:23.791301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:24.019182 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:24.019150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:24.306156 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:24.306074    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:24.402348 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:24.402318    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:25.620389 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.620363    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:31:25.697935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.697895    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-certs\") pod \"de93880d-134d-45e8-9c07-d6ac2a789adb\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") "
Jul 28 18:31:25.698136 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.697974    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcgpf\" (UniqueName: \"kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-kube-api-access-zcgpf\") pod \"de93880d-134d-45e8-9c07-d6ac2a789adb\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") "
Jul 28 18:31:25.698136 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698010    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-data\") pod \"de93880d-134d-45e8-9c07-d6ac2a789adb\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") "
Jul 28 18:31:25.698136 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698047    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-envoy\") pod \"de93880d-134d-45e8-9c07-d6ac2a789adb\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") "
Jul 28 18:31:25.698136 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698095    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-credential-socket\") pod \"de93880d-134d-45e8-9c07-d6ac2a789adb\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") "
Jul 28 18:31:25.698136 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698131    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de93880d-134d-45e8-9c07-d6ac2a789adb-istiod-ca-cert\") pod \"de93880d-134d-45e8-9c07-d6ac2a789adb\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") "
Jul 28 18:31:25.698401 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698161    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-podinfo\") pod \"de93880d-134d-45e8-9c07-d6ac2a789adb\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") "
Jul 28 18:31:25.698401 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698217    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-token\") pod \"de93880d-134d-45e8-9c07-d6ac2a789adb\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") "
Jul 28 18:31:25.698401 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698207    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "de93880d-134d-45e8-9c07-d6ac2a789adb" (UID: "de93880d-134d-45e8-9c07-d6ac2a789adb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:25.698401 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698273    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-socket\") pod \"de93880d-134d-45e8-9c07-d6ac2a789adb\" (UID: \"de93880d-134d-45e8-9c07-d6ac2a789adb\") "
Jul 28 18:31:25.698608 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698448    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "de93880d-134d-45e8-9c07-d6ac2a789adb" (UID: "de93880d-134d-45e8-9c07-d6ac2a789adb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:25.698608 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698543    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de93880d-134d-45e8-9c07-d6ac2a789adb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "de93880d-134d-45e8-9c07-d6ac2a789adb" (UID: "de93880d-134d-45e8-9c07-d6ac2a789adb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:31:25.698713 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698669    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "de93880d-134d-45e8-9c07-d6ac2a789adb" (UID: "de93880d-134d-45e8-9c07-d6ac2a789adb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:25.698767 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698748    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:25.698848 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698765    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:25.698848 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698780    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de93880d-134d-45e8-9c07-d6ac2a789adb-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:25.698962 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.698882    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-data" (OuterVolumeSpecName: "istio-data") pod "de93880d-134d-45e8-9c07-d6ac2a789adb" (UID: "de93880d-134d-45e8-9c07-d6ac2a789adb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:25.700357 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.700328    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-kube-api-access-zcgpf" (OuterVolumeSpecName: "kube-api-access-zcgpf") pod "de93880d-134d-45e8-9c07-d6ac2a789adb" (UID: "de93880d-134d-45e8-9c07-d6ac2a789adb"). InnerVolumeSpecName "kube-api-access-zcgpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:31:25.700479 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.700461    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "de93880d-134d-45e8-9c07-d6ac2a789adb" (UID: "de93880d-134d-45e8-9c07-d6ac2a789adb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:31:25.700738 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.700707    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "de93880d-134d-45e8-9c07-d6ac2a789adb" (UID: "de93880d-134d-45e8-9c07-d6ac2a789adb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:25.700738 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.700715    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-token" (OuterVolumeSpecName: "istio-token") pod "de93880d-134d-45e8-9c07-d6ac2a789adb" (UID: "de93880d-134d-45e8-9c07-d6ac2a789adb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:31:25.800187 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.800153    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:25.800187 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.800184    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:25.800187 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.800193    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:25.800426 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.800203    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcgpf\" (UniqueName: \"kubernetes.io/projected/de93880d-134d-45e8-9c07-d6ac2a789adb-kube-api-access-zcgpf\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:25.800426 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.800212    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:25.800426 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:25.800220    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de93880d-134d-45e8-9c07-d6ac2a789adb-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:26.203675 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.203638    2583 generic.go:356] "Generic (PLEG): container finished" podID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerID="716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327" exitCode=0
Jul 28 18:31:26.203868 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.203748    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"
Jul 28 18:31:26.203868 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.203776    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" event={"ID":"de93880d-134d-45e8-9c07-d6ac2a789adb","Type":"ContainerDied","Data":"716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327"}
Jul 28 18:31:26.203957 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.203866    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh" event={"ID":"de93880d-134d-45e8-9c07-d6ac2a789adb","Type":"ContainerDied","Data":"900e97feaa27dfb953777dca1fd2ebf5743f1316c2179ba37189dd2234b55d9a"}
Jul 28 18:31:26.203957 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.203896    2583 scope.go:117] "RemoveContainer" containerID="716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327"
Jul 28 18:31:26.216939 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.216802    2583 scope.go:117] "RemoveContainer" containerID="5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7"
Jul 28 18:31:26.225496 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.225473    2583 scope.go:117] "RemoveContainer" containerID="716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327"
Jul 28 18:31:26.225776 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:31:26.225756    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327\": container with ID starting with 716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327 not found: ID does not exist" containerID="716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327"
Jul 28 18:31:26.225870 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.225785    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327"} err="failed to get container status \"716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327\": rpc error: code = NotFound desc = could not find container \"716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327\": container with ID starting with 716b99f182bdb69dc267936ae3eac78712cc82e7b6dc0e7454938fb78380b327 not found: ID does not exist"
Jul 28 18:31:26.225870 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.225803    2583 scope.go:117] "RemoveContainer" containerID="5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7"
Jul 28 18:31:26.226039 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:31:26.226023    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7\": container with ID starting with 5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7 not found: ID does not exist" containerID="5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7"
Jul 28 18:31:26.226096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.226044    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7"} err="failed to get container status \"5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7\": rpc error: code = NotFound desc = could not find container \"5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7\": container with ID starting with 5099b7fc2d81bb8d37280a8a316a414bd71f00be0e3935cef4426223c4d917d7 not found: ID does not exist"
Jul 28 18:31:26.227529 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.227511    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"]
Jul 28 18:31:26.233683 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:26.233654    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b13f7e6e-openshift-default-65cb55f94b-qsbvh"]
Jul 28 18:31:27.411236 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:27.411202    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" path="/var/lib/kubelet/pods/de93880d-134d-45e8-9c07-d6ac2a789adb/volumes"
Jul 28 18:31:29.467955 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:29.467920    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:29.568509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:29.568475    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:34.621033 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:34.620993    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:31:37.266666 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:37.266628    2583 generic.go:356] "Generic (PLEG): container finished" podID="6852c94b-9494-4b6b-955e-8a8681342d79" containerID="6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440" exitCode=0
Jul 28 18:31:37.267191 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:37.266702    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" event={"ID":"6852c94b-9494-4b6b-955e-8a8681342d79","Type":"ContainerDied","Data":"6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440"}
Jul 28 18:31:37.267191 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:37.267142    2583 scope.go:117] "RemoveContainer" containerID="6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440"
Jul 28 18:31:37.268191 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:37.268166    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:31:37.268276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:37.268231    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:31:37.268276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:37.268263    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:31:38.186247 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:38.186211    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:38.273569 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:38.273534    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" event={"ID":"6852c94b-9494-4b6b-955e-8a8681342d79","Type":"ContainerStarted","Data":"3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82"}
Jul 28 18:31:38.414553 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:38.414517    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:38.415899 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:38.415870    2583 patch_prober.go:28] interesting pod/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 28 18:31:38.416028 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:38.415921    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 28 18:31:39.414071 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:39.414035    2583 patch_prober.go:28] interesting pod/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 28 18:31:39.414483 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:39.414088    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 28 18:31:40.414519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:40.414484    2583 patch_prober.go:28] interesting pod/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 28 18:31:40.414962 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:40.414538    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 28 18:31:41.414599 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:41.414563    2583 patch_prober.go:28] interesting pod/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 28 18:31:41.415104 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:41.414636    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 28 18:31:42.417793 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:42.417760    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:42.418233 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:42.418105    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:42.418791 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:42.418773    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:43.330096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:43.330029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:43.394432 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:43.394400    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:43.451619 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:43.451584    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:43.577551 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:43.577523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:43.708202 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:43.708163    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"]
Jul 28 18:31:43.719701 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:43.719675    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:43.939165 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:43.939129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:44.304974 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:44.304943    2583 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" secret="" err="secret \"e2e-derive-12084f6d-openshift-default-dockercfg-xzxlt\" not found"
Jul 28 18:31:44.418865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:44.418807    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:44.519796 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:44.519766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:44.619954 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:44.619871    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:45.310118 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:45.310079    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy" containerID="cri-o://3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82" gracePeriod=30
Jul 28 18:31:45.996151 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:45.996121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:47.895328 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:47.895294    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:31:50.576267 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.576193    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:50.644069 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644034    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-envoy\") pod \"6852c94b-9494-4b6b-955e-8a8681342d79\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") "
Jul 28 18:31:50.644238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644099    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-certs\") pod \"6852c94b-9494-4b6b-955e-8a8681342d79\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") "
Jul 28 18:31:50.644238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644131    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-socket\") pod \"6852c94b-9494-4b6b-955e-8a8681342d79\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") "
Jul 28 18:31:50.644238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644162    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6852c94b-9494-4b6b-955e-8a8681342d79-istio-podinfo\") pod \"6852c94b-9494-4b6b-955e-8a8681342d79\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") "
Jul 28 18:31:50.644238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644195    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-istio-token\") pod \"6852c94b-9494-4b6b-955e-8a8681342d79\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") "
Jul 28 18:31:50.644238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644222    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6852c94b-9494-4b6b-955e-8a8681342d79-istiod-ca-cert\") pod \"6852c94b-9494-4b6b-955e-8a8681342d79\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") "
Jul 28 18:31:50.644539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644257    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-data\") pod \"6852c94b-9494-4b6b-955e-8a8681342d79\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") "
Jul 28 18:31:50.644539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644346    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wgvl\" (UniqueName: \"kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-kube-api-access-6wgvl\") pod \"6852c94b-9494-4b6b-955e-8a8681342d79\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") "
Jul 28 18:31:50.644539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644395    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-credential-socket\") pod \"6852c94b-9494-4b6b-955e-8a8681342d79\" (UID: \"6852c94b-9494-4b6b-955e-8a8681342d79\") "
Jul 28 18:31:50.644539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644440    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6852c94b-9494-4b6b-955e-8a8681342d79" (UID: "6852c94b-9494-4b6b-955e-8a8681342d79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:50.644539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644468    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6852c94b-9494-4b6b-955e-8a8681342d79" (UID: "6852c94b-9494-4b6b-955e-8a8681342d79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:50.644785 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644667    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6852c94b-9494-4b6b-955e-8a8681342d79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6852c94b-9494-4b6b-955e-8a8681342d79" (UID: "6852c94b-9494-4b6b-955e-8a8681342d79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:31:50.644785 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644779    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:50.644964 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644879    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6852c94b-9494-4b6b-955e-8a8681342d79" (UID: "6852c94b-9494-4b6b-955e-8a8681342d79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:50.644964 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644898    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:50.644964 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.644924    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6852c94b-9494-4b6b-955e-8a8681342d79-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:50.645320 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.645299    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-data" (OuterVolumeSpecName: "istio-data") pod "6852c94b-9494-4b6b-955e-8a8681342d79" (UID: "6852c94b-9494-4b6b-955e-8a8681342d79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:50.646429 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.646400    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6852c94b-9494-4b6b-955e-8a8681342d79" (UID: "6852c94b-9494-4b6b-955e-8a8681342d79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:31:50.646951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.646919    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-kube-api-access-6wgvl" (OuterVolumeSpecName: "kube-api-access-6wgvl") pod "6852c94b-9494-4b6b-955e-8a8681342d79" (UID: "6852c94b-9494-4b6b-955e-8a8681342d79"). InnerVolumeSpecName "kube-api-access-6wgvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:31:50.647051 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.646991    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-istio-token" (OuterVolumeSpecName: "istio-token") pod "6852c94b-9494-4b6b-955e-8a8681342d79" (UID: "6852c94b-9494-4b6b-955e-8a8681342d79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:31:50.647106 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.647042    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6852c94b-9494-4b6b-955e-8a8681342d79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6852c94b-9494-4b6b-955e-8a8681342d79" (UID: "6852c94b-9494-4b6b-955e-8a8681342d79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:31:50.745990 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.745958    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wgvl\" (UniqueName: \"kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-kube-api-access-6wgvl\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:50.745990 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.745987    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:50.745990 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.745997    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:50.746237 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.746009    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6852c94b-9494-4b6b-955e-8a8681342d79-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:50.746237 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.746019    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6852c94b-9494-4b6b-955e-8a8681342d79-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:50.746237 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:50.746031    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6852c94b-9494-4b6b-955e-8a8681342d79-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:31:51.348050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.348011    2583 generic.go:356] "Generic (PLEG): container finished" podID="6852c94b-9494-4b6b-955e-8a8681342d79" containerID="3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82" exitCode=0
Jul 28 18:31:51.348238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.348070    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"
Jul 28 18:31:51.348238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.348109    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" event={"ID":"6852c94b-9494-4b6b-955e-8a8681342d79","Type":"ContainerDied","Data":"3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82"}
Jul 28 18:31:51.348238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.348178    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq" event={"ID":"6852c94b-9494-4b6b-955e-8a8681342d79","Type":"ContainerDied","Data":"befbd58be422658231fadf03b9fb265bcfd9ce82171e9e359294cac4911b14e8"}
Jul 28 18:31:51.348238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.348209    2583 scope.go:117] "RemoveContainer" containerID="3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82"
Jul 28 18:31:51.357640 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.357619    2583 scope.go:117] "RemoveContainer" containerID="6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440"
Jul 28 18:31:51.365537 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.365520    2583 scope.go:117] "RemoveContainer" containerID="3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82"
Jul 28 18:31:51.365770 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:31:51.365751    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82\": container with ID starting with 3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82 not found: ID does not exist" containerID="3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82"
Jul 28 18:31:51.365867 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.365783    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82"} err="failed to get container status \"3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82\": rpc error: code = NotFound desc = could not find container \"3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82\": container with ID starting with 3f7429881681ad9a09f9964e077d4d04127cc040ebca5309eb7378837ec45a82 not found: ID does not exist"
Jul 28 18:31:51.365867 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.365809    2583 scope.go:117] "RemoveContainer" containerID="6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440"
Jul 28 18:31:51.366085 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:31:51.366064    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440\": container with ID starting with 6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440 not found: ID does not exist" containerID="6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440"
Jul 28 18:31:51.366125 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.366094    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440"} err="failed to get container status \"6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440\": rpc error: code = NotFound desc = could not find container \"6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440\": container with ID starting with 6fddc599d23b576c06866028cfcd58866eb2aa1425127bb0b67afd443cd8f440 not found: ID does not exist"
Jul 28 18:31:51.376180 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.376144    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"]
Jul 28 18:31:51.376335 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.376201    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-12084f6d-openshift-default-56f67f74d4-bfbpq"]
Jul 28 18:31:51.405146 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:51.405112    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" path="/var/lib/kubelet/pods/6852c94b-9494-4b6b-955e-8a8681342d79/volumes"
Jul 28 18:31:52.330508 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:52.330469    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:52.447368 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:52.447328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:52.546073 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:52.546026    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:52.642810 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:52.642723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:53.058149 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:31:53.058110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:19.011580 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:32:19.011487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:25.068375 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:32:25.068341    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:25.178586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:32:25.178558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:25.271799 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:32:25.271754    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:41.150491 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:32:41.150454    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:47.177323 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:32:47.177277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:47.285272 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:32:47.285241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:47.378418 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:32:47.378368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:57.914627 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:32:57.914588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:58.703934 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:32:58.703894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:05.219447 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:05.219414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:05.324075 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:05.324040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:05.420679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:05.420638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:05.521105 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:05.521005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:09.690591 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:09.690551    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:33:10.639734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:10.639704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:10.743859 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:10.743800    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:11.203361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:11.203329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:12.585105 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:12.585069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:17.790884 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:17.790849    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:33:19.076502 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:19.076470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:19.176775 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:19.176743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:19.759506 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:19.759469    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:25.848484 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:25.848448    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:25.948975 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:25.948941    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:26.077851 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.077788    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:26.364781 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.364749    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"]
Jul 28 18:33:26.365150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365135    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy"
Jul 28 18:33:26.365150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365149    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy"
Jul 28 18:33:26.365256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365160    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy"
Jul 28 18:33:26.365256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365166    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy"
Jul 28 18:33:26.365256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365182    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy"
Jul 28 18:33:26.365256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365189    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy"
Jul 28 18:33:26.365256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365195    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy"
Jul 28 18:33:26.365256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365200    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy"
Jul 28 18:33:26.365256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365257    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebfd1db6-0397-4b99-90cc-db3e394f1613" containerName="istio-proxy"
Jul 28 18:33:26.365466 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365265    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy"
Jul 28 18:33:26.365466 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.365271    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy"
Jul 28 18:33:26.368242 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.368224    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.371113 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.371094    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-fec3f09a-openshift-default-dockercfg-76ftr\""
Jul 28 18:33:26.386828 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.386785    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"]
Jul 28 18:33:26.388101 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.388078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:26.466478 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.466440    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-socket\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.466645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.466491    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-data\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.466645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.466551    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fghwp\" (UniqueName: \"kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-kube-api-access-fghwp\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.466645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.466577    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa569534-b559-49c5-8289-2d932b5346d0-istiod-ca-cert\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.466645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.466615    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-istio-token\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.466645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.466633    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-credential-socket\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.466901 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.466673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa569534-b559-49c5-8289-2d932b5346d0-istio-podinfo\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.466901 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.466693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-envoy\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.466901 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.466752    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-certs\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.567800 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.567767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-envoy\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.567841    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-certs\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.567884    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-socket\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.567952    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-data\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.568012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fghwp\" (UniqueName: \"kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-kube-api-access-fghwp\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568257 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.568038    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa569534-b559-49c5-8289-2d932b5346d0-istiod-ca-cert\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568257 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.568099    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-istio-token\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568257 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.568127    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-credential-socket\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568257 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.568160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa569534-b559-49c5-8289-2d932b5346d0-istio-podinfo\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568716 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.568685    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-certs\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.568736    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-socket\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.568765    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa569534-b559-49c5-8289-2d932b5346d0-istiod-ca-cert\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.568778    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-data\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.568853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.568803    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-credential-socket\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.570145 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.570119    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-envoy\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.570459 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.570443    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa569534-b559-49c5-8289-2d932b5346d0-istio-podinfo\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.576534 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.576513    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-istio-token\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.576775 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.576757    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fghwp\" (UniqueName: \"kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-kube-api-access-fghwp\") pod \"e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") " pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.639022 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.638935    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:26.678749 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.678721    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:26.825365 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.825339    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"]
Jul 28 18:33:26.827230 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:33:26.827204    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa569534_b559_49c5_8289_2d932b5346d0.slice/crio-8933381089aac0217ddf3ffa3f20776bc983c489bff9490a93f4d4d2933bd249 WatchSource:0}: Error finding container 8933381089aac0217ddf3ffa3f20776bc983c489bff9490a93f4d4d2933bd249: Status 404 returned error can't find the container with id 8933381089aac0217ddf3ffa3f20776bc983c489bff9490a93f4d4d2933bd249
Jul 28 18:33:26.829400 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.829359    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:33:26.829511 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.829427    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:33:26.829511 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:26.829453    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:33:27.814184 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:27.814148    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" event={"ID":"fa569534-b559-49c5-8289-2d932b5346d0","Type":"ContainerStarted","Data":"d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8"}
Jul 28 18:33:27.814552 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:27.814207    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" event={"ID":"fa569534-b559-49c5-8289-2d932b5346d0","Type":"ContainerStarted","Data":"8933381089aac0217ddf3ffa3f20776bc983c489bff9490a93f4d4d2933bd249"}
Jul 28 18:33:27.838932 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:27.838875    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" podStartSLOduration=1.838859139 podStartE2EDuration="1.838859139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:33:27.83703679 +0000 UTC m=+1958.913598995" watchObservedRunningTime="2026-07-28 18:33:27.838859139 +0000 UTC m=+1958.915421135"
Jul 28 18:33:28.679441 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:28.679406    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:28.684067 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:28.684042    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:28.818959 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:28.818926    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:28.819785 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:28.819765    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:29.725477 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:29.725414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:29.824256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:29.824223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:29.973951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:29.973917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:30.011767 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:30.011684    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:33:30.518577 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:30.518543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:30.608434 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:30.608405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:30.723551 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:30.723522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:41.789016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:41.788979    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:33:43.820090 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:43.820064    2583 patch_prober.go:28] interesting pod/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 18:33:43.820414 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:43.820128    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 18:33:43.894295 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:43.894262    2583 generic.go:356] "Generic (PLEG): container finished" podID="fa569534-b559-49c5-8289-2d932b5346d0" containerID="d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8" exitCode=0
Jul 28 18:33:43.894461 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:43.894333    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" event={"ID":"fa569534-b559-49c5-8289-2d932b5346d0","Type":"ContainerDied","Data":"d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8"}
Jul 28 18:33:43.894711 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:43.894697    2583 scope.go:117] "RemoveContainer" containerID="d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8"
Jul 28 18:33:43.895783 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:43.895756    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:33:43.895900 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:43.895844    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:33:43.895900 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:43.895878    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:33:44.900563 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:44.900527    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" event={"ID":"fa569534-b559-49c5-8289-2d932b5346d0","Type":"ContainerStarted","Data":"67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f"}
Jul 28 18:33:45.678936 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:45.678903    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:45.679129 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:45.679044    2583 patch_prober.go:28] interesting pod/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 18:33:45.679129 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:45.679104    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 18:33:46.679688 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:46.679645    2583 patch_prober.go:28] interesting pod/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 18:33:46.680112 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:46.679724    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 18:33:47.679664 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:47.679631    2583 patch_prober.go:28] interesting pod/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 18:33:47.679856 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:47.679686    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 18:33:48.679599 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:48.679560    2583 patch_prober.go:28] interesting pod/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 28 18:33:48.679777 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:48.679629    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 28 18:33:49.683282 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:49.683251    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:49.683646 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:49.683595    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:49.684235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:49.684214    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:33:54.752361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:54.752330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:55.445542 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:55.445491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:55.535590 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:55.535564    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:33:56.899408 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:33:56.899374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:01.642199 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:01.642163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:01.701545 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:01.701511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:12.510641 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:12.510611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:12.568431 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:12.568395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:12.922003 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:12.921917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:15.094205 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:15.094174    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:34:16.626494 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:16.626466    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:19.126377 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.126325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:19.226954 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.226911    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:19.373477 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.373438    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"]
Jul 28 18:34:19.373704 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.373678    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy" containerID="cri-o://67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f" gracePeriod=30
Jul 28 18:34:19.394044 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.393949    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:19.665552 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.665460    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:19.685997 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.685968    2583 patch_prober.go:28] interesting pod/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 18:34:19.686138 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.686019    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 18:34:19.753555 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.753519    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:19.852689 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.852657    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:19.989905 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:19.989851    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:24.628961 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.628931    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:34:24.726673 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.726639    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fghwp\" (UniqueName: \"kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-kube-api-access-fghwp\") pod \"fa569534-b559-49c5-8289-2d932b5346d0\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") "
Jul 28 18:34:24.726875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.726687    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-socket\") pod \"fa569534-b559-49c5-8289-2d932b5346d0\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") "
Jul 28 18:34:24.726875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.726800    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-envoy\") pod \"fa569534-b559-49c5-8289-2d932b5346d0\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") "
Jul 28 18:34:24.726875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.726863    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-certs\") pod \"fa569534-b559-49c5-8289-2d932b5346d0\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") "
Jul 28 18:34:24.726992 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.726927    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-istio-token\") pod \"fa569534-b559-49c5-8289-2d932b5346d0\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") "
Jul 28 18:34:24.726992 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.726956    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa569534-b559-49c5-8289-2d932b5346d0-istio-podinfo\") pod \"fa569534-b559-49c5-8289-2d932b5346d0\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") "
Jul 28 18:34:24.726992 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.726985    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-data\") pod \"fa569534-b559-49c5-8289-2d932b5346d0\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") "
Jul 28 18:34:24.727126 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.727028    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa569534-b559-49c5-8289-2d932b5346d0-istiod-ca-cert\") pod \"fa569534-b559-49c5-8289-2d932b5346d0\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") "
Jul 28 18:34:24.727126 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.727034    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fa569534-b559-49c5-8289-2d932b5346d0" (UID: "fa569534-b559-49c5-8289-2d932b5346d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:34:24.727126 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.727055    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-credential-socket\") pod \"fa569534-b559-49c5-8289-2d932b5346d0\" (UID: \"fa569534-b559-49c5-8289-2d932b5346d0\") "
Jul 28 18:34:24.727358 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.727153    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fa569534-b559-49c5-8289-2d932b5346d0" (UID: "fa569534-b559-49c5-8289-2d932b5346d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:34:24.727431 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.727344    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fa569534-b559-49c5-8289-2d932b5346d0" (UID: "fa569534-b559-49c5-8289-2d932b5346d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:34:24.727487 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.727446    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:34:24.727487 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.727468    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:34:24.727597 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.727485    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:34:24.727597 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.727503    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa569534-b559-49c5-8289-2d932b5346d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fa569534-b559-49c5-8289-2d932b5346d0" (UID: "fa569534-b559-49c5-8289-2d932b5346d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:34:24.727784 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.727760    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "fa569534-b559-49c5-8289-2d932b5346d0" (UID: "fa569534-b559-49c5-8289-2d932b5346d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:34:24.729053 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.729030    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fa569534-b559-49c5-8289-2d932b5346d0" (UID: "fa569534-b559-49c5-8289-2d932b5346d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:34:24.729158 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.729060    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-kube-api-access-fghwp" (OuterVolumeSpecName: "kube-api-access-fghwp") pod "fa569534-b559-49c5-8289-2d932b5346d0" (UID: "fa569534-b559-49c5-8289-2d932b5346d0"). InnerVolumeSpecName "kube-api-access-fghwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:34:24.729158 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.729062    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa569534-b559-49c5-8289-2d932b5346d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fa569534-b559-49c5-8289-2d932b5346d0" (UID: "fa569534-b559-49c5-8289-2d932b5346d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:34:24.729401 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.729383    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa569534-b559-49c5-8289-2d932b5346d0" (UID: "fa569534-b559-49c5-8289-2d932b5346d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:34:24.828870 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.828803    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:34:24.828870 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.828865    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa569534-b559-49c5-8289-2d932b5346d0-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:34:24.828870 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.828875    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fghwp\" (UniqueName: \"kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-kube-api-access-fghwp\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:34:24.829116 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.828884    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa569534-b559-49c5-8289-2d932b5346d0-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:34:24.829116 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.828893    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa569534-b559-49c5-8289-2d932b5346d0-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:34:24.829116 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:24.828901    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa569534-b559-49c5-8289-2d932b5346d0-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:34:25.095019 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.094933    2583 generic.go:356] "Generic (PLEG): container finished" podID="fa569534-b559-49c5-8289-2d932b5346d0" containerID="67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f" exitCode=0
Jul 28 18:34:25.095019 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.094982    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"
Jul 28 18:34:25.095019 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.094995    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" event={"ID":"fa569534-b559-49c5-8289-2d932b5346d0","Type":"ContainerDied","Data":"67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f"}
Jul 28 18:34:25.095268 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.095037    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk" event={"ID":"fa569534-b559-49c5-8289-2d932b5346d0","Type":"ContainerDied","Data":"8933381089aac0217ddf3ffa3f20776bc983c489bff9490a93f4d4d2933bd249"}
Jul 28 18:34:25.095268 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.095059    2583 scope.go:117] "RemoveContainer" containerID="67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f"
Jul 28 18:34:25.104115 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.104097    2583 scope.go:117] "RemoveContainer" containerID="d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8"
Jul 28 18:34:25.112146 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.112127    2583 scope.go:117] "RemoveContainer" containerID="67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f"
Jul 28 18:34:25.112434 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:34:25.112415    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f\": container with ID starting with 67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f not found: ID does not exist" containerID="67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f"
Jul 28 18:34:25.112477 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.112443    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f"} err="failed to get container status \"67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f\": rpc error: code = NotFound desc = could not find container \"67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f\": container with ID starting with 67db6bdfe9ecd40a820ca4e10c2cafabe155942cc75cb1a2fd1c58fbaa66024f not found: ID does not exist"
Jul 28 18:34:25.112477 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.112461    2583 scope.go:117] "RemoveContainer" containerID="d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8"
Jul 28 18:34:25.112703 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:34:25.112686    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8\": container with ID starting with d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8 not found: ID does not exist" containerID="d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8"
Jul 28 18:34:25.112744 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.112710    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8"} err="failed to get container status \"d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8\": rpc error: code = NotFound desc = could not find container \"d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8\": container with ID starting with d404c6bd86046f5d85623a6536217d9b99423b3ce858ab5cc7290308efbe49d8 not found: ID does not exist"
Jul 28 18:34:25.117179 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.117158    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"]
Jul 28 18:34:25.120186 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.120165    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-fec3f09a-openshift-default-64596fd5b4-fcxbk"]
Jul 28 18:34:25.406060 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:25.405976    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa569534-b559-49c5-8289-2d932b5346d0" path="/var/lib/kubelet/pods/fa569534-b559-49c5-8289-2d932b5346d0/volumes"
Jul 28 18:34:26.060694 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:26.060648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:26.118276 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:26.118238    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:26.203157 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:26.203123    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:26.295657 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:26.295629    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:26.428731 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:26.428646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:32.169872 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:32.169839    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:32.228241 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:32.228211    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:35.281862 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:35.281809    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:39.128755 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.128723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:39.418787 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.418701    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"]
Jul 28 18:34:39.419170 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.419151    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy"
Jul 28 18:34:39.419170 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.419171    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy"
Jul 28 18:34:39.419291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.419197    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy"
Jul 28 18:34:39.419291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.419203    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy"
Jul 28 18:34:39.419291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.419263    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy"
Jul 28 18:34:39.419291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.419274    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa569534-b559-49c5-8289-2d932b5346d0" containerName="istio-proxy"
Jul 28 18:34:39.419291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.419280    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="de93880d-134d-45e8-9c07-d6ac2a789adb" containerName="istio-proxy"
Jul 28 18:34:39.419291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.419285    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6852c94b-9494-4b6b-955e-8a8681342d79" containerName="istio-proxy"
Jul 28 18:34:39.422249 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.422230    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.425424 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.425403    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4cfa56f3-openshift-default-dockercfg-wsd2x\""
Jul 28 18:34:39.431916 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.431895    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"]
Jul 28 18:34:39.438944 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.438918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:39.466246 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.466207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-certs\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.466448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.466254    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jf64\" (UniqueName: \"kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-kube-api-access-7jf64\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.466448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.466273    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6803f782-8051-4ae8-97ab-28608652f7a8-istiod-ca-cert\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.466448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.466306    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6803f782-8051-4ae8-97ab-28608652f7a8-istio-podinfo\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.466448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.466372    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-envoy\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.466636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.466460    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-data\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.466636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.466486    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-istio-token\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.466636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.466519    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-credential-socket\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.466733 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.466694    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-socket\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.567638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.567603    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-socket\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.567836 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.567646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-certs\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.567836 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.567680    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jf64\" (UniqueName: \"kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-kube-api-access-7jf64\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.567836 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.567710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6803f782-8051-4ae8-97ab-28608652f7a8-istiod-ca-cert\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.567836 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.567754    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6803f782-8051-4ae8-97ab-28608652f7a8-istio-podinfo\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.567836 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.567792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-envoy\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.568128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.567882    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-data\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.568128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.567907    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-istio-token\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.568128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.568017    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-credential-socket\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.568291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.568164    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-socket\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.568291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.568261    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-data\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.568421 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.568275    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-credential-socket\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.568421 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.568402    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-certs\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.568526 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.568480    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6803f782-8051-4ae8-97ab-28608652f7a8-istiod-ca-cert\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.570002 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.569982    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-envoy\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.570223 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.570203    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6803f782-8051-4ae8-97ab-28608652f7a8-istio-podinfo\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.579835 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.579796    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jf64\" (UniqueName: \"kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-kube-api-access-7jf64\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.579959 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.579805    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-istio-token\") pod \"e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") " pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.619083 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.619052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:39.733926 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.733893    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:39.876559 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.876532    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"]
Jul 28 18:34:39.879077 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:34:39.879049    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6803f782_8051_4ae8_97ab_28608652f7a8.slice/crio-3a09daac8cd6b373acde8f5001e9d26bddec755af3aaa2160656fa8f1458b2cc WatchSource:0}: Error finding container 3a09daac8cd6b373acde8f5001e9d26bddec755af3aaa2160656fa8f1458b2cc: Status 404 returned error can't find the container with id 3a09daac8cd6b373acde8f5001e9d26bddec755af3aaa2160656fa8f1458b2cc
Jul 28 18:34:39.881548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.881517    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:34:39.881638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.881596    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:34:39.881638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:39.881626    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:34:40.171118 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:40.171028    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" event={"ID":"6803f782-8051-4ae8-97ab-28608652f7a8","Type":"ContainerStarted","Data":"be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757"}
Jul 28 18:34:40.171118 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:40.171097    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" event={"ID":"6803f782-8051-4ae8-97ab-28608652f7a8","Type":"ContainerStarted","Data":"3a09daac8cd6b373acde8f5001e9d26bddec755af3aaa2160656fa8f1458b2cc"}
Jul 28 18:34:40.192648 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:40.192598    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" podStartSLOduration=1.192582892 podStartE2EDuration="1.192582892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:34:40.190370479 +0000 UTC m=+2031.266932476" watchObservedRunningTime="2026-07-28 18:34:40.192582892 +0000 UTC m=+2031.269144889"
Jul 28 18:34:40.734921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:40.734879    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:40.740040 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:40.740018    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:41.176596 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:41.176512    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:41.177566 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:41.177544    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:42.123117 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:42.123081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:42.194675 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:42.194636    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:42.248926 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:42.248895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:42.579029 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:42.578992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:42.774956 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:42.774920    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:42.854953 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:42.854872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:34:47.489603 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:47.489558    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:34:55.252915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:55.252879    2583 generic.go:356] "Generic (PLEG): container finished" podID="6803f782-8051-4ae8-97ab-28608652f7a8" containerID="be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757" exitCode=0
Jul 28 18:34:55.253327 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:55.252930    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" event={"ID":"6803f782-8051-4ae8-97ab-28608652f7a8","Type":"ContainerDied","Data":"be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757"}
Jul 28 18:34:55.253381 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:55.253351    2583 scope.go:117] "RemoveContainer" containerID="be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757"
Jul 28 18:34:55.254414 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:55.254388    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:34:55.254474 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:55.254452    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:34:55.254511 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:55.254482    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:34:56.177181 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:56.177142    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:56.262072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:56.262032    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" event={"ID":"6803f782-8051-4ae8-97ab-28608652f7a8","Type":"ContainerStarted","Data":"4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe"}
Jul 28 18:34:56.736375 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:56.735759    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:34:56.736375 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:56.735904    2583 patch_prober.go:28] interesting pod/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 18:34:56.736375 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:56.735966    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 18:34:57.735243 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:57.735206    2583 patch_prober.go:28] interesting pod/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 18:34:57.735849 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:57.735799    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 18:34:58.735031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:58.734550    2583 patch_prober.go:28] interesting pod/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 18:34:58.735031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:58.734617    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 18:34:59.735325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:59.735276    2583 patch_prober.go:28] interesting pod/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 28 18:34:59.735845 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:34:59.735353    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 28 18:35:00.739000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:00.738969    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:35:00.739424 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:00.739298    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:35:00.740074 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:00.740060    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:35:07.143332 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:07.143296    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:35:16.691109 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.691032    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"]
Jul 28 18:35:16.694922 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.694903    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.697826 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.697794    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4cfa56f3\"/\"kube-root-ca.crt\""
Jul 28 18:35:16.699320 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.699298    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4cfa56f3\"/\"openshift-service-ca.crt\""
Jul 28 18:35:16.699630 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.699606    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4cfa56f3\"/\"e2e-lifecycle-model-4cfa56f3-kserve-self-signed-certs\""
Jul 28 18:35:16.699730 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.699656    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4cfa56f3\"/\"default-dockercfg-tdq75\""
Jul 28 18:35:16.704038 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.704016    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"]
Jul 28 18:35:16.709006 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.708984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:35:16.738241 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.738204    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tmp-dir\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.738241 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.738244    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tls-certs\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.738448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.738266    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-model-cache\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.738448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.738284    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-home\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.738448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.738330    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-dshm\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.738448 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.738394    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crtsc\" (UniqueName: \"kubernetes.io/projected/db7db478-81c5-4cae-ba22-badbf3f2f6e7-kube-api-access-crtsc\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.839882 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.839848    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crtsc\" (UniqueName: \"kubernetes.io/projected/db7db478-81c5-4cae-ba22-badbf3f2f6e7-kube-api-access-crtsc\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.840107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.839929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tmp-dir\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.840107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.839975    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tls-certs\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.840107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.840000    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-model-cache\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.840107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.840031    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-home\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.840107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.840063    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-dshm\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.840437 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.840414    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tmp-dir\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.840437 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.840430    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-model-cache\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.840547 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.840485    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-home\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.842289 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.842270    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-dshm\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.842623 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.842607    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tls-certs\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:16.848408 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:16.848385    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crtsc\" (UniqueName: \"kubernetes.io/projected/db7db478-81c5-4cae-ba22-badbf3f2f6e7-kube-api-access-crtsc\") pod \"e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") " pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:17.004546 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:17.004454    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:17.169555 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:17.169525    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"]
Jul 28 18:35:17.172530 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:35:17.172498    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb7db478_81c5_4cae_ba22_badbf3f2f6e7.slice/crio-b4af67ed307d22853a9592a0846ab3f7c8e8dde0eb5d6cba934cfc43d79c6658 WatchSource:0}: Error finding container b4af67ed307d22853a9592a0846ab3f7c8e8dde0eb5d6cba934cfc43d79c6658: Status 404 returned error can't find the container with id b4af67ed307d22853a9592a0846ab3f7c8e8dde0eb5d6cba934cfc43d79c6658
Jul 28 18:35:17.367488 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:17.367392    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm" event={"ID":"db7db478-81c5-4cae-ba22-badbf3f2f6e7","Type":"ContainerStarted","Data":"909aa920ed331acaece7533df0a9176437edf914b1700ab2268218ea85f632a4"}
Jul 28 18:35:17.367488 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:17.367435    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm" event={"ID":"db7db478-81c5-4cae-ba22-badbf3f2f6e7","Type":"ContainerStarted","Data":"b4af67ed307d22853a9592a0846ab3f7c8e8dde0eb5d6cba934cfc43d79c6658"}
Jul 28 18:35:17.386354 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:17.386297    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm" podStartSLOduration=1.386279927 podStartE2EDuration="1.386279927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:35:17.384912477 +0000 UTC m=+2068.461474478" watchObservedRunningTime="2026-07-28 18:35:17.386279927 +0000 UTC m=+2068.462841924"
Jul 28 18:35:19.786426 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:19.786390    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:35:27.005255 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:27.005201    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:27.005255 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:27.005263    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:27.017458 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:27.017429    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:27.428201 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:27.428120    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:35:39.393868 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:39.393832    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:35:49.485513 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:49.485474    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:35:49.488852 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:35:49.488809    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:36:00.793214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:00.793180    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:36:02.161193 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:02.161162    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:02.221237 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:02.221205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:02.286898 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:02.286871    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:02.448882 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:02.448852    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:05.487215 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:05.487181    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:10.392985 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:10.392950    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:36:10.467963 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:10.467930    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"]
Jul 28 18:36:10.511023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:10.510990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:25.487470 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.487409    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm" podUID="db7db478-81c5-4cae-ba22-badbf3f2f6e7" containerName="main" containerID="cri-o://909aa920ed331acaece7533df0a9176437edf914b1700ab2268218ea85f632a4" gracePeriod=45
Jul 28 18:36:25.705513 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.705482    2583 generic.go:356] "Generic (PLEG): container finished" podID="db7db478-81c5-4cae-ba22-badbf3f2f6e7" containerID="909aa920ed331acaece7533df0a9176437edf914b1700ab2268218ea85f632a4" exitCode=0
Jul 28 18:36:25.705683 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.705557    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm" event={"ID":"db7db478-81c5-4cae-ba22-badbf3f2f6e7","Type":"ContainerDied","Data":"909aa920ed331acaece7533df0a9176437edf914b1700ab2268218ea85f632a4"}
Jul 28 18:36:25.737316 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.737292    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:36:25.833098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.832994    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tmp-dir\") pod \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") "
Jul 28 18:36:25.833098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833059    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crtsc\" (UniqueName: \"kubernetes.io/projected/db7db478-81c5-4cae-ba22-badbf3f2f6e7-kube-api-access-crtsc\") pod \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") "
Jul 28 18:36:25.833098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833078    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-home\") pod \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") "
Jul 28 18:36:25.833387 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833131    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-dshm\") pod \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") "
Jul 28 18:36:25.833387 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833222    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tls-certs\") pod \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") "
Jul 28 18:36:25.833387 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833258    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-model-cache\") pod \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\" (UID: \"db7db478-81c5-4cae-ba22-badbf3f2f6e7\") "
Jul 28 18:36:25.833387 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833312    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db7db478-81c5-4cae-ba22-badbf3f2f6e7" (UID: "db7db478-81c5-4cae-ba22-badbf3f2f6e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:36:25.833387 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833365    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-home" (OuterVolumeSpecName: "home") pod "db7db478-81c5-4cae-ba22-badbf3f2f6e7" (UID: "db7db478-81c5-4cae-ba22-badbf3f2f6e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:36:25.833625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833536    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-model-cache" (OuterVolumeSpecName: "model-cache") pod "db7db478-81c5-4cae-ba22-badbf3f2f6e7" (UID: "db7db478-81c5-4cae-ba22-badbf3f2f6e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:36:25.833695 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833677    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tmp-dir\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:25.833733 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833700    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-home\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:25.833733 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.833712    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-model-cache\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:25.835375 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.835346    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db7db478-81c5-4cae-ba22-badbf3f2f6e7-kube-api-access-crtsc" (OuterVolumeSpecName: "kube-api-access-crtsc") pod "db7db478-81c5-4cae-ba22-badbf3f2f6e7" (UID: "db7db478-81c5-4cae-ba22-badbf3f2f6e7"). InnerVolumeSpecName "kube-api-access-crtsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:36:25.835375 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.835366    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-dshm" (OuterVolumeSpecName: "dshm") pod "db7db478-81c5-4cae-ba22-badbf3f2f6e7" (UID: "db7db478-81c5-4cae-ba22-badbf3f2f6e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:36:25.835551 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.835376    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db7db478-81c5-4cae-ba22-badbf3f2f6e7" (UID: "db7db478-81c5-4cae-ba22-badbf3f2f6e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:36:25.934944 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.934902    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crtsc\" (UniqueName: \"kubernetes.io/projected/db7db478-81c5-4cae-ba22-badbf3f2f6e7-kube-api-access-crtsc\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:25.934944 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.934937    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db7db478-81c5-4cae-ba22-badbf3f2f6e7-dshm\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:25.935156 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:25.934955    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db7db478-81c5-4cae-ba22-badbf3f2f6e7-tls-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:26.717322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:26.717295    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"
Jul 28 18:36:26.717876 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:26.717285    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm" event={"ID":"db7db478-81c5-4cae-ba22-badbf3f2f6e7","Type":"ContainerDied","Data":"b4af67ed307d22853a9592a0846ab3f7c8e8dde0eb5d6cba934cfc43d79c6658"}
Jul 28 18:36:26.717876 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:26.717433    2583 scope.go:117] "RemoveContainer" containerID="909aa920ed331acaece7533df0a9176437edf914b1700ab2268218ea85f632a4"
Jul 28 18:36:26.738902 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:26.738877    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"]
Jul 28 18:36:26.742330 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:26.742303    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-4cfa56f3/e2e-lifecycle-model-4cfa56f3-kserve-685b85bcf-vrflm"]
Jul 28 18:36:27.406047 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:27.406018    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db7db478-81c5-4cae-ba22-badbf3f2f6e7" path="/var/lib/kubelet/pods/db7db478-81c5-4cae-ba22-badbf3f2f6e7/volumes"
Jul 28 18:36:46.743250 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:46.743165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:47.955322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:47.955291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:53.537980 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:53.537938    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:53.599441 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:53.599410    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:53.803386 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:53.803298    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"]
Jul 28 18:36:53.803617 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:53.803589    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy" containerID="cri-o://4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe" gracePeriod=30
Jul 28 18:36:53.819792 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:53.819760    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:54.058092 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:54.058007    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:55.076609 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:55.076557    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:55.176407 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:55.176369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:55.279686 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:55.279647    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:55.980677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:55.980643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:56.037976 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:56.037948    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:56.105016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:56.104987    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:56.503382 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:56.503348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:57.549318 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:57.549278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:36:57.695017 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:57.694985    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:36:59.068970 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.068897    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:36:59.150595 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.150561    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-credential-socket\") pod \"6803f782-8051-4ae8-97ab-28608652f7a8\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") "
Jul 28 18:36:59.150775 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.150606    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-envoy\") pod \"6803f782-8051-4ae8-97ab-28608652f7a8\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") "
Jul 28 18:36:59.150775 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.150636    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-istio-token\") pod \"6803f782-8051-4ae8-97ab-28608652f7a8\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") "
Jul 28 18:36:59.150775 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.150685    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-certs\") pod \"6803f782-8051-4ae8-97ab-28608652f7a8\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") "
Jul 28 18:36:59.150775 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.150706    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-socket\") pod \"6803f782-8051-4ae8-97ab-28608652f7a8\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") "
Jul 28 18:36:59.150775 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.150753    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6803f782-8051-4ae8-97ab-28608652f7a8-istiod-ca-cert\") pod \"6803f782-8051-4ae8-97ab-28608652f7a8\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") "
Jul 28 18:36:59.151039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.150792    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jf64\" (UniqueName: \"kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-kube-api-access-7jf64\") pod \"6803f782-8051-4ae8-97ab-28608652f7a8\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") "
Jul 28 18:36:59.151039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.150841    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-data\") pod \"6803f782-8051-4ae8-97ab-28608652f7a8\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") "
Jul 28 18:36:59.151039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.150863    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6803f782-8051-4ae8-97ab-28608652f7a8-istio-podinfo\") pod \"6803f782-8051-4ae8-97ab-28608652f7a8\" (UID: \"6803f782-8051-4ae8-97ab-28608652f7a8\") "
Jul 28 18:36:59.151039 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.150883    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6803f782-8051-4ae8-97ab-28608652f7a8" (UID: "6803f782-8051-4ae8-97ab-28608652f7a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:36:59.151245 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.151088    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6803f782-8051-4ae8-97ab-28608652f7a8" (UID: "6803f782-8051-4ae8-97ab-28608652f7a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:36:59.151245 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.151148    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6803f782-8051-4ae8-97ab-28608652f7a8" (UID: "6803f782-8051-4ae8-97ab-28608652f7a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:36:59.151245 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.151219    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:59.151245 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.151236    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:59.151454 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.151267    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6803f782-8051-4ae8-97ab-28608652f7a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6803f782-8051-4ae8-97ab-28608652f7a8" (UID: "6803f782-8051-4ae8-97ab-28608652f7a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:36:59.151836 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.151782    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "6803f782-8051-4ae8-97ab-28608652f7a8" (UID: "6803f782-8051-4ae8-97ab-28608652f7a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:36:59.153113 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.153086    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6803f782-8051-4ae8-97ab-28608652f7a8" (UID: "6803f782-8051-4ae8-97ab-28608652f7a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:36:59.153207 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.153091    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6803f782-8051-4ae8-97ab-28608652f7a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6803f782-8051-4ae8-97ab-28608652f7a8" (UID: "6803f782-8051-4ae8-97ab-28608652f7a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:36:59.153351 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.153332    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "6803f782-8051-4ae8-97ab-28608652f7a8" (UID: "6803f782-8051-4ae8-97ab-28608652f7a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:36:59.153410 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.153341    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-kube-api-access-7jf64" (OuterVolumeSpecName: "kube-api-access-7jf64") pod "6803f782-8051-4ae8-97ab-28608652f7a8" (UID: "6803f782-8051-4ae8-97ab-28608652f7a8"). InnerVolumeSpecName "kube-api-access-7jf64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:36:59.252322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.252284    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:59.252322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.252314    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6803f782-8051-4ae8-97ab-28608652f7a8-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:59.252322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.252326    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:59.252572 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.252341    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:59.252572 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.252353    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6803f782-8051-4ae8-97ab-28608652f7a8-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:59.252572 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.252366    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6803f782-8051-4ae8-97ab-28608652f7a8-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:59.252572 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.252378    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jf64\" (UniqueName: \"kubernetes.io/projected/6803f782-8051-4ae8-97ab-28608652f7a8-kube-api-access-7jf64\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:36:59.884776 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.884740    2583 generic.go:356] "Generic (PLEG): container finished" podID="6803f782-8051-4ae8-97ab-28608652f7a8" containerID="4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe" exitCode=0
Jul 28 18:36:59.884973 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.884791    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"
Jul 28 18:36:59.884973 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.884825    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" event={"ID":"6803f782-8051-4ae8-97ab-28608652f7a8","Type":"ContainerDied","Data":"4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe"}
Jul 28 18:36:59.884973 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.884867    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj" event={"ID":"6803f782-8051-4ae8-97ab-28608652f7a8","Type":"ContainerDied","Data":"3a09daac8cd6b373acde8f5001e9d26bddec755af3aaa2160656fa8f1458b2cc"}
Jul 28 18:36:59.884973 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.884888    2583 scope.go:117] "RemoveContainer" containerID="4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe"
Jul 28 18:36:59.893429 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.893411    2583 scope.go:117] "RemoveContainer" containerID="be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757"
Jul 28 18:36:59.901435 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.901379    2583 scope.go:117] "RemoveContainer" containerID="4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe"
Jul 28 18:36:59.901753 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:36:59.901723    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe\": container with ID starting with 4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe not found: ID does not exist" containerID="4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe"
Jul 28 18:36:59.901851 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.901759    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe"} err="failed to get container status \"4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe\": rpc error: code = NotFound desc = could not find container \"4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe\": container with ID starting with 4546131b9abea1586008f5b44c905a48a1d29cddf4ad49ba74f6cf2dd208a6fe not found: ID does not exist"
Jul 28 18:36:59.901851 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.901777    2583 scope.go:117] "RemoveContainer" containerID="be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757"
Jul 28 18:36:59.902053 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:36:59.902035    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757\": container with ID starting with be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757 not found: ID does not exist" containerID="be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757"
Jul 28 18:36:59.902104 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.902057    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757"} err="failed to get container status \"be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757\": rpc error: code = NotFound desc = could not find container \"be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757\": container with ID starting with be8dd1f9bee0ec54c97bf4ea7a300f8697090c2ca9fc87d8eadb12c0fc1aa757 not found: ID does not exist"
Jul 28 18:36:59.904023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.904004    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"]
Jul 28 18:36:59.907547 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:36:59.907529    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4cfa56f3-openshift-default-556bb75dbd-k29pj"]
Jul 28 18:37:01.405603 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:01.405566    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" path="/var/lib/kubelet/pods/6803f782-8051-4ae8-97ab-28608652f7a8/volumes"
Jul 28 18:37:03.425240 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:03.425202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:03.480795 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:03.480762    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:04.115397 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:04.115364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:09.490081 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:09.490044    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:37:10.510085 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:10.510032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:10.611796 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:10.611737    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:10.712281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:10.712234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:21.217055 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:21.217022    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:21.551013 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:21.550904    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:42.558391 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:42.558348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:48.554387 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:48.554352    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:48.654360 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:48.654326    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:48.904629 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:48.904555    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:49.207442 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.207410    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"]
Jul 28 18:37:49.207787 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.207774    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy"
Jul 28 18:37:49.207848 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.207788    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy"
Jul 28 18:37:49.207848 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.207797    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db7db478-81c5-4cae-ba22-badbf3f2f6e7" containerName="main"
Jul 28 18:37:49.207848 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.207803    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="db7db478-81c5-4cae-ba22-badbf3f2f6e7" containerName="main"
Jul 28 18:37:49.207953 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.207849    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy"
Jul 28 18:37:49.207953 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.207856    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy"
Jul 28 18:37:49.207953 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.207913    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="db7db478-81c5-4cae-ba22-badbf3f2f6e7" containerName="main"
Jul 28 18:37:49.207953 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.207922    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy"
Jul 28 18:37:49.207953 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.207930    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6803f782-8051-4ae8-97ab-28608652f7a8" containerName="istio-proxy"
Jul 28 18:37:49.211057 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.211040    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.213991 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.213968    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-61d29d-openshift-default-dockercfg-fmg65\""
Jul 28 18:37:49.223714 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.223693    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"]
Jul 28 18:37:49.228713 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.228681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:49.333251 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.333215    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-data\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.333251 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.333258    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2436d5b4-c624-429e-a0af-62d76647520e-istiod-ca-cert\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.333509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.333281    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nn68\" (UniqueName: \"kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-kube-api-access-4nn68\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.333509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.333303    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-envoy\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.333509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.333365    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-socket\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.333509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.333404    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2436d5b4-c624-429e-a0af-62d76647520e-istio-podinfo\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.333509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.333427    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-istio-token\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.333509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.333444    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-certs\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.333509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.333473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-credential-socket\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.434762 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.434726    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2436d5b4-c624-429e-a0af-62d76647520e-istio-podinfo\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.434762 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.434768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-istio-token\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435049 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.434786    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-certs\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435049 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.434833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-credential-socket\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435049 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.434852    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-data\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435049 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.434876    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2436d5b4-c624-429e-a0af-62d76647520e-istiod-ca-cert\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435049 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.434901    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nn68\" (UniqueName: \"kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-kube-api-access-4nn68\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435049 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.434929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-envoy\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435362 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.434991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-socket\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435362 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.435254    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-credential-socket\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435460 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.435401    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-data\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435521 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.435502    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-socket\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435626 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.435602    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-certs\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.435797 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.435779    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2436d5b4-c624-429e-a0af-62d76647520e-istiod-ca-cert\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.437322 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.437299    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2436d5b4-c624-429e-a0af-62d76647520e-istio-podinfo\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.437473 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.437454    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-envoy\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.442876 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.442856    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-istio-token\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.443105 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.443085    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nn68\" (UniqueName: \"kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-kube-api-access-4nn68\") pod \"e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") " pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.524509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.524424    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-61d29d-openshift-default-dockercfg-fmg65\""
Jul 28 18:37:49.531691 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.531666    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:49.581778 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.581736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:49.690281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.690248    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"]
Jul 28 18:37:49.693270 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:37:49.693237    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2436d5b4_c624_429e_a0af_62d76647520e.slice/crio-13355acbb51e19bb2250f2b4b582056c12473b853fdfdece99429ce344abeb9e WatchSource:0}: Error finding container 13355acbb51e19bb2250f2b4b582056c12473b853fdfdece99429ce344abeb9e: Status 404 returned error can't find the container with id 13355acbb51e19bb2250f2b4b582056c12473b853fdfdece99429ce344abeb9e
Jul 28 18:37:49.695495 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.695462    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:37:49.695605 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.695543    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:37:49.695605 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:49.695586    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:37:50.138490 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:50.138447    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" event={"ID":"2436d5b4-c624-429e-a0af-62d76647520e","Type":"ContainerStarted","Data":"91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684"}
Jul 28 18:37:50.138692 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:50.138512    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" event={"ID":"2436d5b4-c624-429e-a0af-62d76647520e","Type":"ContainerStarted","Data":"13355acbb51e19bb2250f2b4b582056c12473b853fdfdece99429ce344abeb9e"}
Jul 28 18:37:50.160057 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:50.160009    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" podStartSLOduration=1.159993792 podStartE2EDuration="1.159993792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:37:50.158655321 +0000 UTC m=+2221.235217318" watchObservedRunningTime="2026-07-28 18:37:50.159993792 +0000 UTC m=+2221.236555790"
Jul 28 18:37:50.532922 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:50.532886    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:50.537718 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:50.537689    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:51.143623 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:51.143590    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:51.144592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:51.144572    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:37:52.095049 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:52.095016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:52.162461 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:52.162429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:52.227422 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:52.227389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:52.332116 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:52.332084    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:52.763935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:52.763878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:52.864362 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:52.864332    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:37:57.586262 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:37:57.586228    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:38:06.145339 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:06.145302    2583 patch_prober.go:28] interesting pod/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 28 18:38:06.145799 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:06.145359    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 28 18:38:06.219265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:06.219231    2583 generic.go:356] "Generic (PLEG): container finished" podID="2436d5b4-c624-429e-a0af-62d76647520e" containerID="91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684" exitCode=0
Jul 28 18:38:06.219427 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:06.219297    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" event={"ID":"2436d5b4-c624-429e-a0af-62d76647520e","Type":"ContainerDied","Data":"91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684"}
Jul 28 18:38:06.219766 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:06.219744    2583 scope.go:117] "RemoveContainer" containerID="91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684"
Jul 28 18:38:06.220840 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:06.220788    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:38:06.220949 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:06.220879    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:38:06.220949 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:06.220909    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:38:07.225764 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:07.225732    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" event={"ID":"2436d5b4-c624-429e-a0af-62d76647520e","Type":"ContainerStarted","Data":"c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6"}
Jul 28 18:38:07.533319 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:07.532809    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:38:07.533319 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:07.532951    2583 patch_prober.go:28] interesting pod/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 28 18:38:07.533319 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:07.533006    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 28 18:38:08.532846 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:08.532382    2583 patch_prober.go:28] interesting pod/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 28 18:38:08.532846 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:08.532442    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 28 18:38:09.532551 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:09.532504    2583 patch_prober.go:28] interesting pod/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 28 18:38:09.532771 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:09.532595    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 28 18:38:10.532604 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:10.532557    2583 patch_prober.go:28] interesting pod/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 28 18:38:10.533134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:10.532636    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 28 18:38:11.536581 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:11.536493    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:38:11.537026 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:11.536899    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:38:11.537562 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:11.537544    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:38:17.090982 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.090945    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:17.407943 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.407861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:17.677363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.677269    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"]
Jul 28 18:38:17.679922 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.679901    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.682932 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.682915    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-db5d01-openshift-default-dockercfg-sq8fr\""
Jul 28 18:38:17.691148 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.691127    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"]
Jul 28 18:38:17.697412 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.697392    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:17.717619 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.717590    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-credential-socket\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.717777 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.717626    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-envoy\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.717777 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.717666    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-certs\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.717914 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.717768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-podinfo\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.717914 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.717828    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-data\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.717914 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.717871    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/870242a6-561c-4ba2-a837-21d664e7c1ac-istiod-ca-cert\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.717914 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.717906    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-socket\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.718061 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.717924    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2d64\" (UniqueName: \"kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-kube-api-access-j2d64\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.718061 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.717948    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-token\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819084 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819046    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-envoy\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819106    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-certs\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819154    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-podinfo\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819176    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-data\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819200    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/870242a6-561c-4ba2-a837-21d664e7c1ac-istiod-ca-cert\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819227    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-socket\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819250    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2d64\" (UniqueName: \"kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-kube-api-access-j2d64\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819289    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-token\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819601 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819339    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-credential-socket\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819702 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819603    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-certs\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819702 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819681    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-credential-socket\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819794 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819713    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-socket\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.819991 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.819966    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-data\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.820107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.820008    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/870242a6-561c-4ba2-a837-21d664e7c1ac-istiod-ca-cert\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.821925 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.821898    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-envoy\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.822243 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.822215    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-podinfo\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.829003 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.828978    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-token\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.829108 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.829084    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2d64\" (UniqueName: \"kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-kube-api-access-j2d64\") pod \"e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") " pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.991710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.991670    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:17.994205 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:17.994172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:18.132028 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:18.132005    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"]
Jul 28 18:38:18.134454 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:38:18.134418    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod870242a6_561c_4ba2_a837_21d664e7c1ac.slice/crio-571ffcb4c3a871951b1831b0c1cf52a103eb25e9a3dd4c273d803b7c070ffa77 WatchSource:0}: Error finding container 571ffcb4c3a871951b1831b0c1cf52a103eb25e9a3dd4c273d803b7c070ffa77: Status 404 returned error can't find the container with id 571ffcb4c3a871951b1831b0c1cf52a103eb25e9a3dd4c273d803b7c070ffa77
Jul 28 18:38:18.136618 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:18.136585    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:38:18.136719 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:18.136667    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:38:18.136719 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:18.136712    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:38:18.285395 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:18.285312    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" event={"ID":"870242a6-561c-4ba2-a837-21d664e7c1ac","Type":"ContainerStarted","Data":"eccbc2a65896f7c1f03e3813c323c39b2410050671600b1a5930dccf3e85933e"}
Jul 28 18:38:18.285395 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:18.285362    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" event={"ID":"870242a6-561c-4ba2-a837-21d664e7c1ac","Type":"ContainerStarted","Data":"571ffcb4c3a871951b1831b0c1cf52a103eb25e9a3dd4c273d803b7c070ffa77"}
Jul 28 18:38:18.309327 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:18.309269    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" podStartSLOduration=1.309248399 podStartE2EDuration="1.309248399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:38:18.3089937 +0000 UTC m=+2249.385555720" watchObservedRunningTime="2026-07-28 18:38:18.309248399 +0000 UTC m=+2249.385810398"
Jul 28 18:38:18.992545 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:18.992506    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:19.997063 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:19.997036    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:20.210889 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:20.210828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:20.295434 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:20.295334    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:20.297680 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:20.297652    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:20.298737 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:20.298717    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:20.356290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:20.356260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:20.849390 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:20.849354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:20.941316 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:20.941257    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:21.103208 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:21.103100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:27.591860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:27.591826    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:38:35.633056 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:35.633010    2583 patch_prober.go:28] interesting pod/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": read tcp 10.132.0.2:40178->10.132.0.50:15021: read: connection reset by peer" start-of-body=
Jul 28 18:38:35.633522 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:35.633115    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": read tcp 10.132.0.2:40178->10.132.0.50:15021: read: connection reset by peer"
Jul 28 18:38:36.390333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:36.390294    2583 generic.go:356] "Generic (PLEG): container finished" podID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerID="eccbc2a65896f7c1f03e3813c323c39b2410050671600b1a5930dccf3e85933e" exitCode=0
Jul 28 18:38:36.390523 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:36.390355    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" event={"ID":"870242a6-561c-4ba2-a837-21d664e7c1ac","Type":"ContainerDied","Data":"eccbc2a65896f7c1f03e3813c323c39b2410050671600b1a5930dccf3e85933e"}
Jul 28 18:38:36.390874 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:36.390857    2583 scope.go:117] "RemoveContainer" containerID="eccbc2a65896f7c1f03e3813c323c39b2410050671600b1a5930dccf3e85933e"
Jul 28 18:38:36.391973 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:36.391943    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:38:36.392057 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:36.392013    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:38:36.392057 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:36.392050    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:38:37.405721 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:37.405684    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" event={"ID":"870242a6-561c-4ba2-a837-21d664e7c1ac","Type":"ContainerStarted","Data":"cc7e97e0f9ea6f7a0f981a305c916aa5c5f709cb8f86f4e147dfe7067cee6322"}
Jul 28 18:38:37.991926 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:37.991887    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:37.992161 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:37.992137    2583 patch_prober.go:28] interesting pod/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 28 18:38:37.992244 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:37.992206    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 28 18:38:38.993026 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:38.992989    2583 patch_prober.go:28] interesting pod/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 28 18:38:38.993430 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:38.993048    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 28 18:38:39.992073 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:39.992040    2583 patch_prober.go:28] interesting pod/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 28 18:38:39.992266 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:39.992094    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 28 18:38:40.992629 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:40.992594    2583 patch_prober.go:28] interesting pod/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 28 18:38:40.993076 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:40.992646    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 28 18:38:41.995780 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:41.995743    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:41.996235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:41.996096    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:41.996656 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:41.996641    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:38:45.261795 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:45.261762    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:52.952557 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:52.952520    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:53.019876 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.019838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:53.501591 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.501559    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"]
Jul 28 18:38:53.504717 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.504698    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.508087 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.508062    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-61d29d\"/\"test-model-61d29d-kserve-self-signed-certs\""
Jul 28 18:38:53.508087 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.508064    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-61d29d\"/\"default-dockercfg-vs6z9\""
Jul 28 18:38:53.508299 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.508125    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-61d29d\"/\"kube-root-ca.crt\""
Jul 28 18:38:53.508528 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.508515    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-61d29d\"/\"openshift-service-ca.crt\""
Jul 28 18:38:53.516699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.516679    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"]
Jul 28 18:38:53.570478 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.570445    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:38:53.585441 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.585413    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-model-cache\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.585585 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.585446    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-dshm\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.585585 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.585470    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg9f7\" (UniqueName: \"kubernetes.io/projected/2ec29f3a-6a98-466d-b9f2-26f22748bbca-kube-api-access-cg9f7\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.585585 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.585555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-home\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.585721 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.585588    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tls-certs\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.585721 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.585633    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tmp-dir\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.686740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.686700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-home\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.686960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.686748    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tls-certs\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.686960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.686777    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tmp-dir\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.686960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.686870    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-model-cache\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.686960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.686891    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-dshm\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.686960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.686922    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg9f7\" (UniqueName: \"kubernetes.io/projected/2ec29f3a-6a98-466d-b9f2-26f22748bbca-kube-api-access-cg9f7\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.687209 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.687174    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-home\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.687267 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.687245    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tmp-dir\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.687330 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.687291    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-model-cache\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.689052 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.689031    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-dshm\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.689390 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.689375    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tls-certs\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.694758 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.694740    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg9f7\" (UniqueName: \"kubernetes.io/projected/2ec29f3a-6a98-466d-b9f2-26f22748bbca-kube-api-access-cg9f7\") pod \"test-model-61d29d-kserve-7579bc5b6f-rfxhq\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") " pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.816319 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.816215    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:38:53.953376 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:53.953179    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"]
Jul 28 18:38:53.956229 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:38:53.956205    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ec29f3a_6a98_466d_b9f2_26f22748bbca.slice/crio-60bf7f198f7ab7b72f0d76d15edbe4935e72381574a68e5ea6476f560dfc994c WatchSource:0}: Error finding container 60bf7f198f7ab7b72f0d76d15edbe4935e72381574a68e5ea6476f560dfc994c: Status 404 returned error can't find the container with id 60bf7f198f7ab7b72f0d76d15edbe4935e72381574a68e5ea6476f560dfc994c
Jul 28 18:38:54.493457 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:54.493419    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq" event={"ID":"2ec29f3a-6a98-466d-b9f2-26f22748bbca","Type":"ContainerStarted","Data":"1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b"}
Jul 28 18:38:54.493668 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:54.493472    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq" event={"ID":"2ec29f3a-6a98-466d-b9f2-26f22748bbca","Type":"ContainerStarted","Data":"60bf7f198f7ab7b72f0d76d15edbe4935e72381574a68e5ea6476f560dfc994c"}
Jul 28 18:38:54.510977 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:54.510921    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq" podStartSLOduration=1.510903446 podStartE2EDuration="1.510903446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:38:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:38:54.509841041 +0000 UTC m=+2285.586403037" watchObservedRunningTime="2026-07-28 18:38:54.510903446 +0000 UTC m=+2285.587465443"
Jul 28 18:38:57.088535 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:38:57.088493    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:39:03.817034 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:03.816989    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:39:03.817034 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:03.817046    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:39:03.829840 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:03.829786    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:39:04.557375 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:04.557348    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:39:08.089624 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:08.089584    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:39:20.118506 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.118472    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"]
Jul 28 18:39:20.122028 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.122003    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.127430 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.127406    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-db5d01\"/\"test-model-db5d01-kserve-self-signed-certs\""
Jul 28 18:39:20.127564 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.127427    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-db5d01\"/\"openshift-service-ca.crt\""
Jul 28 18:39:20.127564 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.127458    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-db5d01\"/\"kube-root-ca.crt\""
Jul 28 18:39:20.127564 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.127461    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-db5d01\"/\"default-dockercfg-r79md\""
Jul 28 18:39:20.132789 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.132768    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"]
Jul 28 18:39:20.142019 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.141998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:39:20.160142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.160120    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42nps\" (UniqueName: \"kubernetes.io/projected/9258123c-b54e-4ab2-8b02-16524e8eba4c-kube-api-access-42nps\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.160238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.160160    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9258123c-b54e-4ab2-8b02-16524e8eba4c-tls-certs\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.160238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.160204    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-home\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.160332 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.160262    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-model-cache\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.160373 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.160352    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-tmp-dir\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.160435 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.160417    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-dshm\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.261031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.261001    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-tmp-dir\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.261200 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.261050    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-dshm\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.261200 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.261090    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42nps\" (UniqueName: \"kubernetes.io/projected/9258123c-b54e-4ab2-8b02-16524e8eba4c-kube-api-access-42nps\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.261200 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.261107    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9258123c-b54e-4ab2-8b02-16524e8eba4c-tls-certs\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.261200 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.261140    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-home\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.261200 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.261166    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-model-cache\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.261480 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.261444    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-tmp-dir\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.261539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.261497    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-home\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.261585 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.261563    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-model-cache\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.263363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.263337    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-dshm\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.263609 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.263593    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9258123c-b54e-4ab2-8b02-16524e8eba4c-tls-certs\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.270240 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.270214    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42nps\" (UniqueName: \"kubernetes.io/projected/9258123c-b54e-4ab2-8b02-16524e8eba4c-kube-api-access-42nps\") pod \"test-model-db5d01-kserve-5c48cb76b5-r74vd\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") " pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.432551 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.432467    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:20.573726 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.573696    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"]
Jul 28 18:39:20.575922 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:39:20.575888    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9258123c_b54e_4ab2_8b02_16524e8eba4c.slice/crio-64044ba7e2f8711dd38e6ae865e29d69825d70868116fdc61ac09a9abd2da625 WatchSource:0}: Error finding container 64044ba7e2f8711dd38e6ae865e29d69825d70868116fdc61ac09a9abd2da625: Status 404 returned error can't find the container with id 64044ba7e2f8711dd38e6ae865e29d69825d70868116fdc61ac09a9abd2da625
Jul 28 18:39:20.625554 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:20.625530    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd" event={"ID":"9258123c-b54e-4ab2-8b02-16524e8eba4c","Type":"ContainerStarted","Data":"64044ba7e2f8711dd38e6ae865e29d69825d70868116fdc61ac09a9abd2da625"}
Jul 28 18:39:21.632285 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:21.632234    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd" event={"ID":"9258123c-b54e-4ab2-8b02-16524e8eba4c","Type":"ContainerStarted","Data":"dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e"}
Jul 28 18:39:21.652517 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:21.652472    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd" podStartSLOduration=1.652457837 podStartE2EDuration="1.652457837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:39:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:39:21.649667805 +0000 UTC m=+2312.726229803" watchObservedRunningTime="2026-07-28 18:39:21.652457837 +0000 UTC m=+2312.729019834"
Jul 28 18:39:24.596589 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:24.596554    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:39:30.433266 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:30.433222    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:30.433266 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:30.433276    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:30.447852 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:30.447805    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:30.692319 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:30.692294    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:39:38.095862 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:38.095824    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:39:56.692133 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:56.692027    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:39:56.747810 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:39:56.747774    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:09.955829 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:09.955783    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"]
Jul 28 18:40:09.983537 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:09.983499    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:10.839999 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:10.839956    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"]
Jul 28 18:40:10.859141 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:10.859095    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:14.890705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:14.890671    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:40:21.464178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:21.464145    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:21.531537 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:21.531497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:21.590693 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:21.590654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:21.649289 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:21.649251    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:22.017976 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:22.017946    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:24.976879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:24.976840    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq" podUID="2ec29f3a-6a98-466d-b9f2-26f22748bbca" containerName="main" containerID="cri-o://1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b" gracePeriod=45
Jul 28 18:40:25.236185 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.236114    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:40:25.312859 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.312805    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg9f7\" (UniqueName: \"kubernetes.io/projected/2ec29f3a-6a98-466d-b9f2-26f22748bbca-kube-api-access-cg9f7\") pod \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") "
Jul 28 18:40:25.313032 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.312888    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tmp-dir\") pod \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") "
Jul 28 18:40:25.313032 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.312917    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-model-cache\") pod \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") "
Jul 28 18:40:25.313032 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.312933    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tls-certs\") pod \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") "
Jul 28 18:40:25.313032 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.313013    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-home\") pod \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") "
Jul 28 18:40:25.313230 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.313045    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-dshm\") pod \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\" (UID: \"2ec29f3a-6a98-466d-b9f2-26f22748bbca\") "
Jul 28 18:40:25.313230 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.313170    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-model-cache" (OuterVolumeSpecName: "model-cache") pod "2ec29f3a-6a98-466d-b9f2-26f22748bbca" (UID: "2ec29f3a-6a98-466d-b9f2-26f22748bbca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:25.313230 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.313183    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2ec29f3a-6a98-466d-b9f2-26f22748bbca" (UID: "2ec29f3a-6a98-466d-b9f2-26f22748bbca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:25.313394 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.313277    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-home" (OuterVolumeSpecName: "home") pod "2ec29f3a-6a98-466d-b9f2-26f22748bbca" (UID: "2ec29f3a-6a98-466d-b9f2-26f22748bbca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:25.313394 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.313383    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-home\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.313479 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.313395    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tmp-dir\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.313479 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.313409    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-model-cache\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.315094 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.315065    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ec29f3a-6a98-466d-b9f2-26f22748bbca-kube-api-access-cg9f7" (OuterVolumeSpecName: "kube-api-access-cg9f7") pod "2ec29f3a-6a98-466d-b9f2-26f22748bbca" (UID: "2ec29f3a-6a98-466d-b9f2-26f22748bbca"). InnerVolumeSpecName "kube-api-access-cg9f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:25.315200 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.315109    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2ec29f3a-6a98-466d-b9f2-26f22748bbca" (UID: "2ec29f3a-6a98-466d-b9f2-26f22748bbca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:40:25.315200 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.315142    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-dshm" (OuterVolumeSpecName: "dshm") pod "2ec29f3a-6a98-466d-b9f2-26f22748bbca" (UID: "2ec29f3a-6a98-466d-b9f2-26f22748bbca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:25.415791 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.415760    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ec29f3a-6a98-466d-b9f2-26f22748bbca-tls-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.415791 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.415792    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ec29f3a-6a98-466d-b9f2-26f22748bbca-dshm\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.416015 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.415808    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg9f7\" (UniqueName: \"kubernetes.io/projected/2ec29f3a-6a98-466d-b9f2-26f22748bbca-kube-api-access-cg9f7\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.860076 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.860016    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd" podUID="9258123c-b54e-4ab2-8b02-16524e8eba4c" containerName="main" containerID="cri-o://dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e" gracePeriod=45
Jul 28 18:40:25.964531 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.964498    2583 generic.go:356] "Generic (PLEG): container finished" podID="2ec29f3a-6a98-466d-b9f2-26f22748bbca" containerID="1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b" exitCode=0
Jul 28 18:40:25.964737 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.964550    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"
Jul 28 18:40:25.964737 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.964576    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq" event={"ID":"2ec29f3a-6a98-466d-b9f2-26f22748bbca","Type":"ContainerDied","Data":"1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b"}
Jul 28 18:40:25.964737 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.964615    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq" event={"ID":"2ec29f3a-6a98-466d-b9f2-26f22748bbca","Type":"ContainerDied","Data":"60bf7f198f7ab7b72f0d76d15edbe4935e72381574a68e5ea6476f560dfc994c"}
Jul 28 18:40:25.964737 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.964632    2583 scope.go:117] "RemoveContainer" containerID="1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b"
Jul 28 18:40:25.973313 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.973285    2583 scope.go:117] "RemoveContainer" containerID="1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b"
Jul 28 18:40:25.973567 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:40:25.973547    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b\": container with ID starting with 1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b not found: ID does not exist" containerID="1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b"
Jul 28 18:40:25.973627 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.973574    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b"} err="failed to get container status \"1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b\": rpc error: code = NotFound desc = could not find container \"1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b\": container with ID starting with 1492758b546fd5318ed5acddf8909f8c673af8119a6724df4012db82f516285b not found: ID does not exist"
Jul 28 18:40:25.985776 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.985748    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"]
Jul 28 18:40:25.989568 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:25.989544    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-61d29d/test-model-61d29d-kserve-7579bc5b6f-rfxhq"]
Jul 28 18:40:26.120303 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.120232    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:40:26.224417 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224376    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-dshm\") pod \"9258123c-b54e-4ab2-8b02-16524e8eba4c\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") "
Jul 28 18:40:26.224599 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224433    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-tmp-dir\") pod \"9258123c-b54e-4ab2-8b02-16524e8eba4c\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") "
Jul 28 18:40:26.224599 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224472    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42nps\" (UniqueName: \"kubernetes.io/projected/9258123c-b54e-4ab2-8b02-16524e8eba4c-kube-api-access-42nps\") pod \"9258123c-b54e-4ab2-8b02-16524e8eba4c\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") "
Jul 28 18:40:26.224599 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224509    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-model-cache\") pod \"9258123c-b54e-4ab2-8b02-16524e8eba4c\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") "
Jul 28 18:40:26.224599 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224553    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9258123c-b54e-4ab2-8b02-16524e8eba4c-tls-certs\") pod \"9258123c-b54e-4ab2-8b02-16524e8eba4c\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") "
Jul 28 18:40:26.224599 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224575    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-home\") pod \"9258123c-b54e-4ab2-8b02-16524e8eba4c\" (UID: \"9258123c-b54e-4ab2-8b02-16524e8eba4c\") "
Jul 28 18:40:26.224857 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224719    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9258123c-b54e-4ab2-8b02-16524e8eba4c" (UID: "9258123c-b54e-4ab2-8b02-16524e8eba4c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:26.224857 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224807    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-model-cache" (OuterVolumeSpecName: "model-cache") pod "9258123c-b54e-4ab2-8b02-16524e8eba4c" (UID: "9258123c-b54e-4ab2-8b02-16524e8eba4c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:26.224980 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224920    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-model-cache\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:26.224980 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224936    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-tmp-dir\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:26.224980 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.224953    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-home" (OuterVolumeSpecName: "home") pod "9258123c-b54e-4ab2-8b02-16524e8eba4c" (UID: "9258123c-b54e-4ab2-8b02-16524e8eba4c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:26.226847 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.226806    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9258123c-b54e-4ab2-8b02-16524e8eba4c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9258123c-b54e-4ab2-8b02-16524e8eba4c" (UID: "9258123c-b54e-4ab2-8b02-16524e8eba4c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:40:26.226847 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.226805    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-dshm" (OuterVolumeSpecName: "dshm") pod "9258123c-b54e-4ab2-8b02-16524e8eba4c" (UID: "9258123c-b54e-4ab2-8b02-16524e8eba4c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:26.227022 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.226894    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9258123c-b54e-4ab2-8b02-16524e8eba4c-kube-api-access-42nps" (OuterVolumeSpecName: "kube-api-access-42nps") pod "9258123c-b54e-4ab2-8b02-16524e8eba4c" (UID: "9258123c-b54e-4ab2-8b02-16524e8eba4c"). InnerVolumeSpecName "kube-api-access-42nps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:26.325485 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.325449    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9258123c-b54e-4ab2-8b02-16524e8eba4c-tls-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:26.325485 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.325480    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-home\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:26.325485 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.325489    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9258123c-b54e-4ab2-8b02-16524e8eba4c-dshm\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:26.325731 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.325497    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42nps\" (UniqueName: \"kubernetes.io/projected/9258123c-b54e-4ab2-8b02-16524e8eba4c-kube-api-access-42nps\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:26.971511 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.971476    2583 generic.go:356] "Generic (PLEG): container finished" podID="9258123c-b54e-4ab2-8b02-16524e8eba4c" containerID="dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e" exitCode=0
Jul 28 18:40:26.971699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.971520    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"
Jul 28 18:40:26.971699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.971568    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd" event={"ID":"9258123c-b54e-4ab2-8b02-16524e8eba4c","Type":"ContainerDied","Data":"dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e"}
Jul 28 18:40:26.971699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.971612    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd" event={"ID":"9258123c-b54e-4ab2-8b02-16524e8eba4c","Type":"ContainerDied","Data":"64044ba7e2f8711dd38e6ae865e29d69825d70868116fdc61ac09a9abd2da625"}
Jul 28 18:40:26.971699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.971634    2583 scope.go:117] "RemoveContainer" containerID="dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e"
Jul 28 18:40:26.983672 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.983479    2583 scope.go:117] "RemoveContainer" containerID="dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e"
Jul 28 18:40:26.983834 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:40:26.983799    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e\": container with ID starting with dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e not found: ID does not exist" containerID="dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e"
Jul 28 18:40:26.983896 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.983843    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e"} err="failed to get container status \"dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e\": rpc error: code = NotFound desc = could not find container \"dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e\": container with ID starting with dfe485e2699be97e5a9d936285b807dd361f7c326eb8070f0e84e73b0c31a87e not found: ID does not exist"
Jul 28 18:40:26.994561 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.994534    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"]
Jul 28 18:40:26.997872 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:26.997843    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-db5d01/test-model-db5d01-kserve-5c48cb76b5-r74vd"]
Jul 28 18:40:27.406195 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:27.406117    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ec29f3a-6a98-466d-b9f2-26f22748bbca" path="/var/lib/kubelet/pods/2ec29f3a-6a98-466d-b9f2-26f22748bbca/volumes"
Jul 28 18:40:27.406548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:27.406535    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9258123c-b54e-4ab2-8b02-16524e8eba4c" path="/var/lib/kubelet/pods/9258123c-b54e-4ab2-8b02-16524e8eba4c/volumes"
Jul 28 18:40:28.164477 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:28.164439    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:28.264942 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:28.264906    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:28.418134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:28.418045    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"]
Jul 28 18:40:28.418323 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:28.418300    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy" containerID="cri-o://c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6" gracePeriod=30
Jul 28 18:40:28.435077 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:28.435038    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:28.674649 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:28.674569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:30.488376 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:30.488338    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:40:33.674261 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.674234    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:40:33.700127 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700096    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-credential-socket\") pod \"2436d5b4-c624-429e-a0af-62d76647520e\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") "
Jul 28 18:40:33.700325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700159    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-data\") pod \"2436d5b4-c624-429e-a0af-62d76647520e\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") "
Jul 28 18:40:33.700325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700185    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-certs\") pod \"2436d5b4-c624-429e-a0af-62d76647520e\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") "
Jul 28 18:40:33.700325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700214    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-envoy\") pod \"2436d5b4-c624-429e-a0af-62d76647520e\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") "
Jul 28 18:40:33.700325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700257    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-socket\") pod \"2436d5b4-c624-429e-a0af-62d76647520e\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") "
Jul 28 18:40:33.700560 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700333    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2436d5b4-c624-429e-a0af-62d76647520e-istiod-ca-cert\") pod \"2436d5b4-c624-429e-a0af-62d76647520e\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") "
Jul 28 18:40:33.700560 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700392    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nn68\" (UniqueName: \"kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-kube-api-access-4nn68\") pod \"2436d5b4-c624-429e-a0af-62d76647520e\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") "
Jul 28 18:40:33.700560 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700415    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-istio-token\") pod \"2436d5b4-c624-429e-a0af-62d76647520e\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") "
Jul 28 18:40:33.700560 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700459    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2436d5b4-c624-429e-a0af-62d76647520e-istio-podinfo\") pod \"2436d5b4-c624-429e-a0af-62d76647520e\" (UID: \"2436d5b4-c624-429e-a0af-62d76647520e\") "
Jul 28 18:40:33.700765 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700630    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2436d5b4-c624-429e-a0af-62d76647520e" (UID: "2436d5b4-c624-429e-a0af-62d76647520e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:33.700765 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700723    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2436d5b4-c624-429e-a0af-62d76647520e" (UID: "2436d5b4-c624-429e-a0af-62d76647520e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:33.700904 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700869    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:33.700904 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.700887    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:33.701223 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.701064    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2436d5b4-c624-429e-a0af-62d76647520e" (UID: "2436d5b4-c624-429e-a0af-62d76647520e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:33.701905 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.701686    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-data" (OuterVolumeSpecName: "istio-data") pod "2436d5b4-c624-429e-a0af-62d76647520e" (UID: "2436d5b4-c624-429e-a0af-62d76647520e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:33.702023 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.701945    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2436d5b4-c624-429e-a0af-62d76647520e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2436d5b4-c624-429e-a0af-62d76647520e" (UID: "2436d5b4-c624-429e-a0af-62d76647520e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:40:33.703213 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.703188    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2436d5b4-c624-429e-a0af-62d76647520e" (UID: "2436d5b4-c624-429e-a0af-62d76647520e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:33.703459 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.703430    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-kube-api-access-4nn68" (OuterVolumeSpecName: "kube-api-access-4nn68") pod "2436d5b4-c624-429e-a0af-62d76647520e" (UID: "2436d5b4-c624-429e-a0af-62d76647520e"). InnerVolumeSpecName "kube-api-access-4nn68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:33.703575 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.703487    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2436d5b4-c624-429e-a0af-62d76647520e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2436d5b4-c624-429e-a0af-62d76647520e" (UID: "2436d5b4-c624-429e-a0af-62d76647520e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:40:33.703653 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.703638    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-istio-token" (OuterVolumeSpecName: "istio-token") pod "2436d5b4-c624-429e-a0af-62d76647520e" (UID: "2436d5b4-c624-429e-a0af-62d76647520e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:33.801957 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.801915    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:33.801957 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.801950    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2436d5b4-c624-429e-a0af-62d76647520e-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:33.802166 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.801968    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nn68\" (UniqueName: \"kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-kube-api-access-4nn68\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:33.802166 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.801982    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2436d5b4-c624-429e-a0af-62d76647520e-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:33.802166 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.801993    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2436d5b4-c624-429e-a0af-62d76647520e-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:33.802166 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.802004    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:33.802166 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:33.802014    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2436d5b4-c624-429e-a0af-62d76647520e-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:34.014399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.014297    2583 generic.go:356] "Generic (PLEG): container finished" podID="2436d5b4-c624-429e-a0af-62d76647520e" containerID="c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6" exitCode=0
Jul 28 18:40:34.014399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.014344    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"
Jul 28 18:40:34.014399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.014381    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" event={"ID":"2436d5b4-c624-429e-a0af-62d76647520e","Type":"ContainerDied","Data":"c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6"}
Jul 28 18:40:34.014646 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.014440    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2" event={"ID":"2436d5b4-c624-429e-a0af-62d76647520e","Type":"ContainerDied","Data":"13355acbb51e19bb2250f2b4b582056c12473b853fdfdece99429ce344abeb9e"}
Jul 28 18:40:34.014646 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.014464    2583 scope.go:117] "RemoveContainer" containerID="c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6"
Jul 28 18:40:34.024991 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.024975    2583 scope.go:117] "RemoveContainer" containerID="91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684"
Jul 28 18:40:34.033195 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.033176    2583 scope.go:117] "RemoveContainer" containerID="c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6"
Jul 28 18:40:34.033483 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:40:34.033460    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6\": container with ID starting with c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6 not found: ID does not exist" containerID="c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6"
Jul 28 18:40:34.033558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.033489    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6"} err="failed to get container status \"c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6\": rpc error: code = NotFound desc = could not find container \"c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6\": container with ID starting with c946e39f7321fa24e0aa791e03a871f807db7b2f855168bf26b16add84ace6e6 not found: ID does not exist"
Jul 28 18:40:34.033558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.033507    2583 scope.go:117] "RemoveContainer" containerID="91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684"
Jul 28 18:40:34.033745 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:40:34.033728    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684\": container with ID starting with 91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684 not found: ID does not exist" containerID="91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684"
Jul 28 18:40:34.033783 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.033749    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684"} err="failed to get container status \"91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684\": rpc error: code = NotFound desc = could not find container \"91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684\": container with ID starting with 91a457d733cb84ce8bf7c4fbe2d73d075b6d5b5c616b9a1322914aedcdc09684 not found: ID does not exist"
Jul 28 18:40:34.038420 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.038393    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"]
Jul 28 18:40:34.041878 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:34.041856    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-61d29d-openshift-default-69c77dbfd-4j8n2"]
Jul 28 18:40:35.405958 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:35.405925    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2436d5b4-c624-429e-a0af-62d76647520e" path="/var/lib/kubelet/pods/2436d5b4-c624-429e-a0af-62d76647520e/volumes"
Jul 28 18:40:39.345343 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:39.345287    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:39.413148 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:39.413114    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:39.469271 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:39.469240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:39.569720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:39.569684    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:39.892551 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:39.892520    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:42.289047 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:42.289011    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:40:45.621677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:45.621643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:45.676072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:45.676032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:45.870602 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:45.870560    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"]
Jul 28 18:40:45.870840 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:45.870796    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy" containerID="cri-o://cc7e97e0f9ea6f7a0f981a305c916aa5c5f709cb8f86f4e147dfe7067cee6322" gracePeriod=30
Jul 28 18:40:45.888549 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:45.888438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:46.112331 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.112299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:46.455405 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.455356    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:46.749547 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749421    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"]
Jul 28 18:40:46.749951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749859    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy"
Jul 28 18:40:46.749951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749875    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy"
Jul 28 18:40:46.749951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749888    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy"
Jul 28 18:40:46.749951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749894    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy"
Jul 28 18:40:46.749951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749901    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ec29f3a-6a98-466d-b9f2-26f22748bbca" containerName="main"
Jul 28 18:40:46.749951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749907    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ec29f3a-6a98-466d-b9f2-26f22748bbca" containerName="main"
Jul 28 18:40:46.749951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749924    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9258123c-b54e-4ab2-8b02-16524e8eba4c" containerName="main"
Jul 28 18:40:46.749951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749929    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9258123c-b54e-4ab2-8b02-16524e8eba4c" containerName="main"
Jul 28 18:40:46.750211 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749986    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy"
Jul 28 18:40:46.750211 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.749993    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ec29f3a-6a98-466d-b9f2-26f22748bbca" containerName="main"
Jul 28 18:40:46.750211 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.750001    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9258123c-b54e-4ab2-8b02-16524e8eba4c" containerName="main"
Jul 28 18:40:46.754857 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.754838    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.757660 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.757640    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ac88b3-openshift-default-dockercfg-tnpc7\""
Jul 28 18:40:46.764582 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.764558    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"]
Jul 28 18:40:46.766260 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.766232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:46.827915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.827880    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-credential-socket\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.828093 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.827925    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-istio-token\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.828093 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.827984    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-socket\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.828093 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.828000    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5538d222-15de-418a-bf45-1876d1ffc867-istio-podinfo\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.828093 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.828056    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njbrb\" (UniqueName: \"kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-kube-api-access-njbrb\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.828093 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.828088    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-data\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.828252 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.828116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-certs\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.828252 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.828153    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-envoy\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.828252 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.828175    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5538d222-15de-418a-bf45-1876d1ffc867-istiod-ca-cert\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929029 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.928993    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-istio-token\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929048    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-socket\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929158    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5538d222-15de-418a-bf45-1876d1ffc867-istio-podinfo\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929212    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njbrb\" (UniqueName: \"kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-kube-api-access-njbrb\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929379 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929236    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-data\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929379 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929264    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-certs\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929379 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929314    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-envoy\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929379 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929346    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5538d222-15de-418a-bf45-1876d1ffc867-istiod-ca-cert\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929379 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929357    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-socket\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929634 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929416    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-credential-socket\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929701 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929674    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-certs\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929753 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929727    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-credential-socket\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.929847 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.929805    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-data\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.930028 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.930008    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5538d222-15de-418a-bf45-1876d1ffc867-istiod-ca-cert\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.931579 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.931555    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-envoy\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.932032 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.932014    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5538d222-15de-418a-bf45-1876d1ffc867-istio-podinfo\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.937411 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.937386    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-istio-token\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:46.937949 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:46.937931    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njbrb\" (UniqueName: \"kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-kube-api-access-njbrb\") pod \"e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") " pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:47.066399 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:47.066318    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:47.193803 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:47.193773    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:47.214994 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:47.214971    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"]
Jul 28 18:40:47.216919 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:40:47.216889    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5538d222_15de_418a_bf45_1876d1ffc867.slice/crio-3e3481c99c0891a21270af5df456ac4a516b7f3ef5fc3bb07c53ecae46144a57 WatchSource:0}: Error finding container 3e3481c99c0891a21270af5df456ac4a516b7f3ef5fc3bb07c53ecae46144a57: Status 404 returned error can't find the container with id 3e3481c99c0891a21270af5df456ac4a516b7f3ef5fc3bb07c53ecae46144a57
Jul 28 18:40:47.219053 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:47.219026    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:40:47.219128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:47.219085    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:40:47.219128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:47.219114    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:40:48.086539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:48.086507    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" event={"ID":"5538d222-15de-418a-bf45-1876d1ffc867","Type":"ContainerStarted","Data":"c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966"}
Jul 28 18:40:48.086961 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:48.086559    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" event={"ID":"5538d222-15de-418a-bf45-1876d1ffc867","Type":"ContainerStarted","Data":"3e3481c99c0891a21270af5df456ac4a516b7f3ef5fc3bb07c53ecae46144a57"}
Jul 28 18:40:48.109918 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:48.109869    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" podStartSLOduration=2.109855205 podStartE2EDuration="2.109855205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:40:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:40:48.107564379 +0000 UTC m=+2399.184126370" watchObservedRunningTime="2026-07-28 18:40:48.109855205 +0000 UTC m=+2399.186417201"
Jul 28 18:40:49.066495 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:49.066455    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:49.071091 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:49.071067    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:49.091982 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:49.091958    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:49.092888 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:49.092869    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:40:49.518261 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:49.518231    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:40:49.521123 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:49.521098    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:40:50.007558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:50.007522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:50.082311 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:50.082274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:50.142535 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:50.142508    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:50.567042 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:50.567003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:50.666901 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:50.666868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:40:51.103764 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.103732    2583 generic.go:356] "Generic (PLEG): container finished" podID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerID="cc7e97e0f9ea6f7a0f981a305c916aa5c5f709cb8f86f4e147dfe7067cee6322" exitCode=0
Jul 28 18:40:51.103969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.103805    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" event={"ID":"870242a6-561c-4ba2-a837-21d664e7c1ac","Type":"ContainerDied","Data":"cc7e97e0f9ea6f7a0f981a305c916aa5c5f709cb8f86f4e147dfe7067cee6322"}
Jul 28 18:40:51.103969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.103896    2583 scope.go:117] "RemoveContainer" containerID="eccbc2a65896f7c1f03e3813c323c39b2410050671600b1a5930dccf3e85933e"
Jul 28 18:40:51.138377 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.138298    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:40:51.274906 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.274864    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-credential-socket\") pod \"870242a6-561c-4ba2-a837-21d664e7c1ac\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") "
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.274922    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/870242a6-561c-4ba2-a837-21d664e7c1ac-istiod-ca-cert\") pod \"870242a6-561c-4ba2-a837-21d664e7c1ac\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") "
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.274950    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-data\") pod \"870242a6-561c-4ba2-a837-21d664e7c1ac\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") "
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.274970    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-socket\") pod \"870242a6-561c-4ba2-a837-21d664e7c1ac\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") "
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.274995    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-envoy\") pod \"870242a6-561c-4ba2-a837-21d664e7c1ac\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") "
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275095    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-podinfo\") pod \"870242a6-561c-4ba2-a837-21d664e7c1ac\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") "
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275135    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "870242a6-561c-4ba2-a837-21d664e7c1ac" (UID: "870242a6-561c-4ba2-a837-21d664e7c1ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275145    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-certs\") pod \"870242a6-561c-4ba2-a837-21d664e7c1ac\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") "
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275244    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "870242a6-561c-4ba2-a837-21d664e7c1ac" (UID: "870242a6-561c-4ba2-a837-21d664e7c1ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275264    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2d64\" (UniqueName: \"kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-kube-api-access-j2d64\") pod \"870242a6-561c-4ba2-a837-21d664e7c1ac\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") "
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275306    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-token\") pod \"870242a6-561c-4ba2-a837-21d664e7c1ac\" (UID: \"870242a6-561c-4ba2-a837-21d664e7c1ac\") "
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275333    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "870242a6-561c-4ba2-a837-21d664e7c1ac" (UID: "870242a6-561c-4ba2-a837-21d664e7c1ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:51.275374 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275355    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/870242a6-561c-4ba2-a837-21d664e7c1ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "870242a6-561c-4ba2-a837-21d664e7c1ac" (UID: "870242a6-561c-4ba2-a837-21d664e7c1ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:40:51.276056 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275651    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:51.276056 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275671    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/870242a6-561c-4ba2-a837-21d664e7c1ac-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:51.276056 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275687    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:51.276056 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275700    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:51.276056 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.275936    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "870242a6-561c-4ba2-a837-21d664e7c1ac" (UID: "870242a6-561c-4ba2-a837-21d664e7c1ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:51.277401 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.277371    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "870242a6-561c-4ba2-a837-21d664e7c1ac" (UID: "870242a6-561c-4ba2-a837-21d664e7c1ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:40:51.277511 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.277416    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "870242a6-561c-4ba2-a837-21d664e7c1ac" (UID: "870242a6-561c-4ba2-a837-21d664e7c1ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:51.277511 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.277441    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "870242a6-561c-4ba2-a837-21d664e7c1ac" (UID: "870242a6-561c-4ba2-a837-21d664e7c1ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:40:51.277594 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.277504    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-kube-api-access-j2d64" (OuterVolumeSpecName: "kube-api-access-j2d64") pod "870242a6-561c-4ba2-a837-21d664e7c1ac" (UID: "870242a6-561c-4ba2-a837-21d664e7c1ac"). InnerVolumeSpecName "kube-api-access-j2d64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:51.376690 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.376655    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:51.376690 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.376684    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:51.376921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.376698    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2d64\" (UniqueName: \"kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-kube-api-access-j2d64\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:51.376921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.376710    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:51.376921 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:51.376721    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/870242a6-561c-4ba2-a837-21d664e7c1ac-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:40:52.110509 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:52.110478    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"
Jul 28 18:40:52.110688 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:52.110529    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw" event={"ID":"870242a6-561c-4ba2-a837-21d664e7c1ac","Type":"ContainerDied","Data":"571ffcb4c3a871951b1831b0c1cf52a103eb25e9a3dd4c273d803b7c070ffa77"}
Jul 28 18:40:52.110688 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:52.110591    2583 scope.go:117] "RemoveContainer" containerID="cc7e97e0f9ea6f7a0f981a305c916aa5c5f709cb8f86f4e147dfe7067cee6322"
Jul 28 18:40:52.129461 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:52.129434    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"]
Jul 28 18:40:52.134283 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:52.134258    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-db5d01-openshift-default-758cd945d8-c2gqw"]
Jul 28 18:40:53.405995 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:53.405958    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" path="/var/lib/kubelet/pods/870242a6-561c-4ba2-a837-21d664e7c1ac/volumes"
Jul 28 18:40:54.396860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:40:54.396823    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:41:02.169781 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:02.169747    2583 generic.go:356] "Generic (PLEG): container finished" podID="5538d222-15de-418a-bf45-1876d1ffc867" containerID="c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966" exitCode=0
Jul 28 18:41:02.170231 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:02.169847    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" event={"ID":"5538d222-15de-418a-bf45-1876d1ffc867","Type":"ContainerDied","Data":"c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966"}
Jul 28 18:41:02.170289 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:02.170257    2583 scope.go:117] "RemoveContainer" containerID="c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966"
Jul 28 18:41:02.171283 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:02.171258    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:41:02.171342 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:02.171327    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:41:02.171387 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:02.171361    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:41:03.177667 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:03.177632    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" event={"ID":"5538d222-15de-418a-bf45-1876d1ffc867","Type":"ContainerStarted","Data":"7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de"}
Jul 28 18:41:04.066744 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:04.066708    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:41:04.068003 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:04.067975    2583 patch_prober.go:28] interesting pod/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 28 18:41:04.068127 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:04.068027    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 28 18:41:04.092423 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:04.092398    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:41:05.067407 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:05.067374    2583 patch_prober.go:28] interesting pod/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 28 18:41:05.067788 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:05.067425    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 28 18:41:06.067705 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:06.067667    2583 patch_prober.go:28] interesting pod/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 28 18:41:06.068126 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:06.067727    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 28 18:41:07.067114 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:07.067074    2583 patch_prober.go:28] interesting pod/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 28 18:41:07.067296 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:07.067136    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 28 18:41:08.070805 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:08.070772    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:41:08.071863 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:08.071843    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:41:14.996191 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:14.996094    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:15.476756 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.476725    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:15.757151 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.757075    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"]
Jul 28 18:41:15.757585 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.757520    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy"
Jul 28 18:41:15.757585 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.757545    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy"
Jul 28 18:41:15.757758 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.757722    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy"
Jul 28 18:41:15.757758 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.757737    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2436d5b4-c624-429e-a0af-62d76647520e" containerName="istio-proxy"
Jul 28 18:41:15.757758 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.757746    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy"
Jul 28 18:41:15.760326 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.760307    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.763710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.763687    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f7f010-openshift-default-dockercfg-zkhcv\""
Jul 28 18:41:15.773169 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.773146    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"]
Jul 28 18:41:15.786969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.786947    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:15.812729 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.812693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-podinfo\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.812933 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.812731    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zrtp\" (UniqueName: \"kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-kube-api-access-5zrtp\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.812933 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.812760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istiod-ca-cert\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.812933 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.812799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-certs\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.812933 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.812873    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-token\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.813118 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.812946    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-credential-socket\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.813118 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.812977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-data\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.813118 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.813027    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-envoy\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.813118 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.813084    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-socket\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.910305 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.910273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:15.914524 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.914496    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-certs\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.914666 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.914540    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-token\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.914666 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.914584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-credential-socket\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.914666 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.914606    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-data\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.914844 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.914672    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-envoy\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.914844 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.914721    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-socket\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.914844 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.914792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-podinfo\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.915012 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.914849    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zrtp\" (UniqueName: \"kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-kube-api-access-5zrtp\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.915012 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.914882    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istiod-ca-cert\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.915012 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.914968    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-certs\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.915168 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.915005    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-credential-socket\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.915227 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.915194    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-socket\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.915313 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.915295    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-data\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.915566 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.915541    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istiod-ca-cert\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.917162 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.917138    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-envoy\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.917287 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.917272    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-podinfo\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.923294 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.923262    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zrtp\" (UniqueName: \"kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-kube-api-access-5zrtp\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:15.923388 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:15.923328    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-token\") pod \"e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") " pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:16.071105 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:16.071016    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:16.207792 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:16.207768    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"]
Jul 28 18:41:16.209877 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:41:16.209851    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9acba892_ca8b_4ac0_a645_47dab29ca9fc.slice/crio-8c5f3d3601f9112a11fc8b56ce02e92bb193271b18fac2e844910a97e87fb1a9 WatchSource:0}: Error finding container 8c5f3d3601f9112a11fc8b56ce02e92bb193271b18fac2e844910a97e87fb1a9: Status 404 returned error can't find the container with id 8c5f3d3601f9112a11fc8b56ce02e92bb193271b18fac2e844910a97e87fb1a9
Jul 28 18:41:16.212014 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:16.211982    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:41:16.212125 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:16.212043    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:41:16.212125 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:16.212070    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:41:16.245829 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:16.245782    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" event={"ID":"9acba892-ca8b-4ac0-a645-47dab29ca9fc","Type":"ContainerStarted","Data":"8c5f3d3601f9112a11fc8b56ce02e92bb193271b18fac2e844910a97e87fb1a9"}
Jul 28 18:41:17.252048 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:17.252011    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" event={"ID":"9acba892-ca8b-4ac0-a645-47dab29ca9fc","Type":"ContainerStarted","Data":"51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37"}
Jul 28 18:41:17.274670 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:17.274611    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" podStartSLOduration=2.274596406 podStartE2EDuration="2.274596406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:41:17.27238993 +0000 UTC m=+2428.348951929" watchObservedRunningTime="2026-07-28 18:41:17.274596406 +0000 UTC m=+2428.351158403"
Jul 28 18:41:18.071780 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:18.071718    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:18.076864 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:18.076836    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:18.257226 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:18.257196    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:18.258321 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:18.258301    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:19.167565 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:19.167534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:19.267032 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:19.267002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:19.399465 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:19.399432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:19.826962 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:19.826934    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:19.930418 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:19.930378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:20.011755 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:20.011716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:22.205687 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:22.205658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:22.262348 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:22.262305    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:23.792880 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:23.792843    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:41:28.934701 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:28.934664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:32.329660 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:32.329625    2583 generic.go:356] "Generic (PLEG): container finished" podID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerID="51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37" exitCode=0
Jul 28 18:41:32.330118 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:32.329677    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" event={"ID":"9acba892-ca8b-4ac0-a645-47dab29ca9fc","Type":"ContainerDied","Data":"51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37"}
Jul 28 18:41:32.330162 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:32.330136    2583 scope.go:117] "RemoveContainer" containerID="51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37"
Jul 28 18:41:32.331355 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:32.331319    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:41:32.331472 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:32.331409    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:41:32.331472 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:32.331447    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:41:32.405555 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:32.405525    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:41:33.257588 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:33.257544    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:33.341364 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:33.341316    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" event={"ID":"9acba892-ca8b-4ac0-a645-47dab29ca9fc","Type":"ContainerStarted","Data":"660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc"}
Jul 28 18:41:34.072142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:34.072103    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:34.073335 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:34.073307    2583 patch_prober.go:28] interesting pod/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 28 18:41:34.073476 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:34.073362    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 28 18:41:35.072330 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:35.072284    2583 patch_prober.go:28] interesting pod/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 28 18:41:35.072716 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:35.072344    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 28 18:41:36.072029 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:36.071994    2583 patch_prober.go:28] interesting pod/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 28 18:41:36.072256 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:36.072236    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 28 18:41:37.071366 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:37.071332    2583 patch_prober.go:28] interesting pod/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 28 18:41:37.071936 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:37.071381    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 28 18:41:38.075456 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:38.075425    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:38.075855 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:38.075776    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:38.076318 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:38.076291    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:41:44.162166 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:41:44.162129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:02.487645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:02.487611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:02.588432 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:02.588393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:02.642253 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:02.642209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:03.022232 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:03.022199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:04.895478 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:04.895445    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:42:08.916549 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:08.916509    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:08.971757 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:08.971720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:09.157576 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:09.157543    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"]
Jul 28 18:42:09.157853 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:09.157801    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy" containerID="cri-o://7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de" gracePeriod=30
Jul 28 18:42:09.177583 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:09.177511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:09.406706 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:09.406675    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:14.408634 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.408613    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:42:14.506768 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.506733    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njbrb\" (UniqueName: \"kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-kube-api-access-njbrb\") pod \"5538d222-15de-418a-bf45-1876d1ffc867\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") "
Jul 28 18:42:14.506958 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.506777    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5538d222-15de-418a-bf45-1876d1ffc867-istiod-ca-cert\") pod \"5538d222-15de-418a-bf45-1876d1ffc867\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") "
Jul 28 18:42:14.506958 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.506873    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-credential-socket\") pod \"5538d222-15de-418a-bf45-1876d1ffc867\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") "
Jul 28 18:42:14.506958 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.506924    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-socket\") pod \"5538d222-15de-418a-bf45-1876d1ffc867\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") "
Jul 28 18:42:14.506958 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.506949    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-envoy\") pod \"5538d222-15de-418a-bf45-1876d1ffc867\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") "
Jul 28 18:42:14.507146 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507014    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-istio-token\") pod \"5538d222-15de-418a-bf45-1876d1ffc867\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") "
Jul 28 18:42:14.507146 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507108    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-certs\") pod \"5538d222-15de-418a-bf45-1876d1ffc867\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") "
Jul 28 18:42:14.507246 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507144    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5538d222-15de-418a-bf45-1876d1ffc867" (UID: "5538d222-15de-418a-bf45-1876d1ffc867"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:42:14.507246 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507181    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5538d222-15de-418a-bf45-1876d1ffc867-istio-podinfo\") pod \"5538d222-15de-418a-bf45-1876d1ffc867\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") "
Jul 28 18:42:14.507246 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507220    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-data\") pod \"5538d222-15de-418a-bf45-1876d1ffc867\" (UID: \"5538d222-15de-418a-bf45-1876d1ffc867\") "
Jul 28 18:42:14.507401 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507297    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5538d222-15de-418a-bf45-1876d1ffc867" (UID: "5538d222-15de-418a-bf45-1876d1ffc867"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:42:14.507401 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507320    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5538d222-15de-418a-bf45-1876d1ffc867" (UID: "5538d222-15de-418a-bf45-1876d1ffc867"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:42:14.507592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507557    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:14.507592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507594    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:14.507791 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507608    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:14.507791 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.507676    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5538d222-15de-418a-bf45-1876d1ffc867-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5538d222-15de-418a-bf45-1876d1ffc867" (UID: "5538d222-15de-418a-bf45-1876d1ffc867"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:42:14.508197 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.508174    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-data" (OuterVolumeSpecName: "istio-data") pod "5538d222-15de-418a-bf45-1876d1ffc867" (UID: "5538d222-15de-418a-bf45-1876d1ffc867"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:42:14.509231 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.509203    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-kube-api-access-njbrb" (OuterVolumeSpecName: "kube-api-access-njbrb") pod "5538d222-15de-418a-bf45-1876d1ffc867" (UID: "5538d222-15de-418a-bf45-1876d1ffc867"). InnerVolumeSpecName "kube-api-access-njbrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:42:14.509336 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.509231    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5538d222-15de-418a-bf45-1876d1ffc867" (UID: "5538d222-15de-418a-bf45-1876d1ffc867"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:42:14.509610 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.509589    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5538d222-15de-418a-bf45-1876d1ffc867-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5538d222-15de-418a-bf45-1876d1ffc867" (UID: "5538d222-15de-418a-bf45-1876d1ffc867"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:42:14.509688 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.509614    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-istio-token" (OuterVolumeSpecName: "istio-token") pod "5538d222-15de-418a-bf45-1876d1ffc867" (UID: "5538d222-15de-418a-bf45-1876d1ffc867"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:42:14.574656 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.574620    2583 generic.go:356] "Generic (PLEG): container finished" podID="5538d222-15de-418a-bf45-1876d1ffc867" containerID="7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de" exitCode=0
Jul 28 18:42:14.574860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.574668    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"
Jul 28 18:42:14.574860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.574702    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" event={"ID":"5538d222-15de-418a-bf45-1876d1ffc867","Type":"ContainerDied","Data":"7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de"}
Jul 28 18:42:14.574860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.574761    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m" event={"ID":"5538d222-15de-418a-bf45-1876d1ffc867","Type":"ContainerDied","Data":"3e3481c99c0891a21270af5df456ac4a516b7f3ef5fc3bb07c53ecae46144a57"}
Jul 28 18:42:14.574860 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.574792    2583 scope.go:117] "RemoveContainer" containerID="7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de"
Jul 28 18:42:14.586664 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.586647    2583 scope.go:117] "RemoveContainer" containerID="c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966"
Jul 28 18:42:14.594300 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.594282    2583 scope.go:117] "RemoveContainer" containerID="7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de"
Jul 28 18:42:14.594538 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:42:14.594520    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de\": container with ID starting with 7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de not found: ID does not exist" containerID="7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de"
Jul 28 18:42:14.594586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.594546    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de"} err="failed to get container status \"7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de\": rpc error: code = NotFound desc = could not find container \"7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de\": container with ID starting with 7572a3d1bedd0586c97d37782607328c2157bbc9b7c49ed04b5824b5af4ca8de not found: ID does not exist"
Jul 28 18:42:14.594586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.594563    2583 scope.go:117] "RemoveContainer" containerID="c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966"
Jul 28 18:42:14.594803 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:42:14.594786    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966\": container with ID starting with c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966 not found: ID does not exist" containerID="c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966"
Jul 28 18:42:14.594874 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.594809    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966"} err="failed to get container status \"c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966\": rpc error: code = NotFound desc = could not find container \"c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966\": container with ID starting with c2a7057b44fa40cce7d322c52c09f8017384d3ec0fbbd973708f9a8ce7bd7966 not found: ID does not exist"
Jul 28 18:42:14.600507 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.600477    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"]
Jul 28 18:42:14.603258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.603238    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ac88b3-openshift-default-7688f67c58-f9h5m"]
Jul 28 18:42:14.608803 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.608784    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:14.608803 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.608804    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:14.608929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.608826    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5538d222-15de-418a-bf45-1876d1ffc867-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:14.608929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.608838    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5538d222-15de-418a-bf45-1876d1ffc867-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:14.608929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.608848    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njbrb\" (UniqueName: \"kubernetes.io/projected/5538d222-15de-418a-bf45-1876d1ffc867-kube-api-access-njbrb\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:14.608929 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:14.608856    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5538d222-15de-418a-bf45-1876d1ffc867-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:15.405983 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:15.405943    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5538d222-15de-418a-bf45-1876d1ffc867" path="/var/lib/kubelet/pods/5538d222-15de-418a-bf45-1876d1ffc867/volumes"
Jul 28 18:42:25.118987 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:25.118955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:25.181247 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:25.181218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:25.238334 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:25.238297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:25.289214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:25.289187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:25.639233 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:25.639202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:29.791376 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:29.791333    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:42:31.330987 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:31.330952    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:31.391206 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:31.391178    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:31.587918 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:31.587840    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"]
Jul 28 18:42:31.588116 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:31.588093    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy" containerID="cri-o://660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc" gracePeriod=30
Jul 28 18:42:31.602463 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:31.602438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:31.818581 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:31.818550    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:32.142057 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.142023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:32.419387 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.419314    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"]
Jul 28 18:42:32.419769 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.419752    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy"
Jul 28 18:42:32.419825 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.419771    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy"
Jul 28 18:42:32.419825 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.419783    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy"
Jul 28 18:42:32.419825 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.419793    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy"
Jul 28 18:42:32.419932 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.419827    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy"
Jul 28 18:42:32.419932 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.419837    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="870242a6-561c-4ba2-a837-21d664e7c1ac" containerName="istio-proxy"
Jul 28 18:42:32.419998 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.419949    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy"
Jul 28 18:42:32.419998 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.419965    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5538d222-15de-418a-bf45-1876d1ffc867" containerName="istio-proxy"
Jul 28 18:42:32.422435 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.422417    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.425185 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.425167    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-37e11d-openshift-default-dockercfg-vd8h9\""
Jul 28 18:42:32.437732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.437703    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"]
Jul 28 18:42:32.438668 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.438638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:32.485889 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.485857    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-socket\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.486035 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.485917    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-credential-socket\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.486035 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.485948    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-certs\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.486035 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.485964    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/319e4143-3d67-48e6-abad-7be053f859e9-istiod-ca-cert\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.486035 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.485986    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/319e4143-3d67-48e6-abad-7be053f859e9-istio-podinfo\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.486170 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.486049    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-envoy\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.486170 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.486103    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-istio-token\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.486170 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.486127    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-data\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.486170 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.486147    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcfmz\" (UniqueName: \"kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-kube-api-access-pcfmz\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.549413 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.549383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:32.587596 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.587562    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-data\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.587746 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.587601    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcfmz\" (UniqueName: \"kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-kube-api-access-pcfmz\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.587746 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.587673    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-socket\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.587746 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.587741    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-credential-socket\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.587939 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.587782    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-certs\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.587939 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.587808    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/319e4143-3d67-48e6-abad-7be053f859e9-istiod-ca-cert\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.587939 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.587873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/319e4143-3d67-48e6-abad-7be053f859e9-istio-podinfo\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.587939 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.587928    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-envoy\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.588430 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.588150    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-data\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.588430 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.587958    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-istio-token\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.588636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.588427    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-socket\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.588686 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.588632    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-credential-socket\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.594598 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.588697    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-certs\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.594598 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.589111    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/319e4143-3d67-48e6-abad-7be053f859e9-istiod-ca-cert\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.594598 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.591539    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-envoy\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.594598 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.591868    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/319e4143-3d67-48e6-abad-7be053f859e9-istio-podinfo\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.597254 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.597227    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-istio-token\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.597908 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.597886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcfmz\" (UniqueName: \"kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-kube-api-access-pcfmz\") pod \"e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") " pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.732348 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.732298    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:32.866652 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.866620    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"]
Jul 28 18:42:32.868580 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:42:32.868552    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod319e4143_3d67_48e6_abad_7be053f859e9.slice/crio-122aa7d03d635c3438714ffd91f4f5ba37ddab148db0f88138932fc68c204e31 WatchSource:0}: Error finding container 122aa7d03d635c3438714ffd91f4f5ba37ddab148db0f88138932fc68c204e31: Status 404 returned error can't find the container with id 122aa7d03d635c3438714ffd91f4f5ba37ddab148db0f88138932fc68c204e31
Jul 28 18:42:32.870765 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.870733    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:42:32.870879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.870801    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:42:32.870879 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:32.870851    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:42:33.670072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:33.670033    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" event={"ID":"319e4143-3d67-48e6-abad-7be053f859e9","Type":"ContainerStarted","Data":"acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25"}
Jul 28 18:42:33.670503 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:33.670088    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" event={"ID":"319e4143-3d67-48e6-abad-7be053f859e9","Type":"ContainerStarted","Data":"122aa7d03d635c3438714ffd91f4f5ba37ddab148db0f88138932fc68c204e31"}
Jul 28 18:42:33.692091 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:33.692039    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" podStartSLOduration=1.692021103 podStartE2EDuration="1.692021103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:42:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:42:33.689921463 +0000 UTC m=+2504.766483460" watchObservedRunningTime="2026-07-28 18:42:33.692021103 +0000 UTC m=+2504.768583099"
Jul 28 18:42:33.732656 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:33.732617    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:33.737646 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:33.737618    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:34.675058 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:34.675026    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:34.675985 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:34.675965    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:35.597042 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:35.597004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:35.687304 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:35.687268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:35.745053 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:35.745019    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:36.248982 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.248950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:36.348969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.348936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:42:36.846377 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.846352    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:42:36.936978 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.936880    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istiod-ca-cert\") pod \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") "
Jul 28 18:42:36.936978 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.936944    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-token\") pod \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") "
Jul 28 18:42:36.937215 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.936995    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-podinfo\") pod \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") "
Jul 28 18:42:36.937215 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.937019    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zrtp\" (UniqueName: \"kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-kube-api-access-5zrtp\") pod \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") "
Jul 28 18:42:36.937215 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.937035    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-certs\") pod \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") "
Jul 28 18:42:36.937215 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.937069    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-data\") pod \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") "
Jul 28 18:42:36.937215 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.937104    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-socket\") pod \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") "
Jul 28 18:42:36.937215 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.937139    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-envoy\") pod \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") "
Jul 28 18:42:36.937215 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.937195    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-credential-socket\") pod \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\" (UID: \"9acba892-ca8b-4ac0-a645-47dab29ca9fc\") "
Jul 28 18:42:36.937594 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.937491    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9acba892-ca8b-4ac0-a645-47dab29ca9fc" (UID: "9acba892-ca8b-4ac0-a645-47dab29ca9fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:42:36.937918 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.937699    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9acba892-ca8b-4ac0-a645-47dab29ca9fc" (UID: "9acba892-ca8b-4ac0-a645-47dab29ca9fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:42:36.937918 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.937714    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9acba892-ca8b-4ac0-a645-47dab29ca9fc" (UID: "9acba892-ca8b-4ac0-a645-47dab29ca9fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:42:36.937918 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.937877    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9acba892-ca8b-4ac0-a645-47dab29ca9fc" (UID: "9acba892-ca8b-4ac0-a645-47dab29ca9fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:42:36.938153 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.938011    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:36.938214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.938172    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "9acba892-ca8b-4ac0-a645-47dab29ca9fc" (UID: "9acba892-ca8b-4ac0-a645-47dab29ca9fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:42:36.939361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.939332    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-kube-api-access-5zrtp" (OuterVolumeSpecName: "kube-api-access-5zrtp") pod "9acba892-ca8b-4ac0-a645-47dab29ca9fc" (UID: "9acba892-ca8b-4ac0-a645-47dab29ca9fc"). InnerVolumeSpecName "kube-api-access-5zrtp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:42:36.939492 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.939367    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "9acba892-ca8b-4ac0-a645-47dab29ca9fc" (UID: "9acba892-ca8b-4ac0-a645-47dab29ca9fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:42:36.939576 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.939552    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9acba892-ca8b-4ac0-a645-47dab29ca9fc" (UID: "9acba892-ca8b-4ac0-a645-47dab29ca9fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:42:36.939725 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:36.939699    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9acba892-ca8b-4ac0-a645-47dab29ca9fc" (UID: "9acba892-ca8b-4ac0-a645-47dab29ca9fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:42:37.038875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.038794    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:37.038875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.038858    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:37.038875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.038867    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:37.038875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.038879    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:37.038875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.038888    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:37.038875 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.038897    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zrtp\" (UniqueName: \"kubernetes.io/projected/9acba892-ca8b-4ac0-a645-47dab29ca9fc-kube-api-access-5zrtp\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:37.039344 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.038906    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:37.039344 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.038913    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9acba892-ca8b-4ac0-a645-47dab29ca9fc-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:42:37.693596 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.693567    2583 generic.go:356] "Generic (PLEG): container finished" podID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerID="660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc" exitCode=0
Jul 28 18:42:37.693759 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.693630    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"
Jul 28 18:42:37.693759 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.693646    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" event={"ID":"9acba892-ca8b-4ac0-a645-47dab29ca9fc","Type":"ContainerDied","Data":"660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc"}
Jul 28 18:42:37.693759 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.693709    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8" event={"ID":"9acba892-ca8b-4ac0-a645-47dab29ca9fc","Type":"ContainerDied","Data":"8c5f3d3601f9112a11fc8b56ce02e92bb193271b18fac2e844910a97e87fb1a9"}
Jul 28 18:42:37.693759 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.693738    2583 scope.go:117] "RemoveContainer" containerID="660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc"
Jul 28 18:42:37.702385 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.702367    2583 scope.go:117] "RemoveContainer" containerID="51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37"
Jul 28 18:42:37.710456 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.710267    2583 scope.go:117] "RemoveContainer" containerID="660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc"
Jul 28 18:42:37.710565 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:42:37.710538    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc\": container with ID starting with 660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc not found: ID does not exist" containerID="660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc"
Jul 28 18:42:37.710610 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.710570    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc"} err="failed to get container status \"660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc\": rpc error: code = NotFound desc = could not find container \"660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc\": container with ID starting with 660d1eb88c513ddc2bdfa4c77380eee7811a91a40115febcdc5e2edeea354ebc not found: ID does not exist"
Jul 28 18:42:37.710610 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.710587    2583 scope.go:117] "RemoveContainer" containerID="51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37"
Jul 28 18:42:37.710842 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:42:37.710804    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37\": container with ID starting with 51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37 not found: ID does not exist" containerID="51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37"
Jul 28 18:42:37.710892 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.710847    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37"} err="failed to get container status \"51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37\": rpc error: code = NotFound desc = could not find container \"51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37\": container with ID starting with 51ec547b862a1e724266660060b9b63adb363a7a3c14a3829a694b9ef7e35e37 not found: ID does not exist"
Jul 28 18:42:37.711526 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.711507    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"]
Jul 28 18:42:37.715204 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:37.715183    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f7f010-openshift-default-647584dc9d-8cjb8"]
Jul 28 18:42:39.406050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:39.406017    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" path="/var/lib/kubelet/pods/9acba892-ca8b-4ac0-a645-47dab29ca9fc/volumes"
Jul 28 18:42:40.787757 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:40.787719    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:42:48.751422 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:48.751386    2583 generic.go:356] "Generic (PLEG): container finished" podID="319e4143-3d67-48e6-abad-7be053f859e9" containerID="acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25" exitCode=0
Jul 28 18:42:48.751865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:48.751434    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" event={"ID":"319e4143-3d67-48e6-abad-7be053f859e9","Type":"ContainerDied","Data":"acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25"}
Jul 28 18:42:48.751865 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:48.751766    2583 scope.go:117] "RemoveContainer" containerID="acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25"
Jul 28 18:42:48.752764 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:48.752736    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:42:48.752836 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:48.752805    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:42:48.752887 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:48.752869    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:42:49.675672 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:49.675652    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:49.761366 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:49.761336    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" event={"ID":"319e4143-3d67-48e6-abad-7be053f859e9","Type":"ContainerStarted","Data":"7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441"}
Jul 28 18:42:50.733483 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:50.733093    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:50.734829 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:50.734461    2583 patch_prober.go:28] interesting pod/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 28 18:42:50.734829 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:50.734520    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" podUID="319e4143-3d67-48e6-abad-7be053f859e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 28 18:42:51.733716 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:51.733676    2583 patch_prober.go:28] interesting pod/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 28 18:42:51.734752 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:51.734305    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" podUID="319e4143-3d67-48e6-abad-7be053f859e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 28 18:42:52.732977 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:52.732936    2583 patch_prober.go:28] interesting pod/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 28 18:42:52.733176 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:52.733020    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" podUID="319e4143-3d67-48e6-abad-7be053f859e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 28 18:42:53.736664 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:53.736635    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:53.737074 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:53.736932    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:42:53.737528 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:42:53.737511    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:43:00.589596 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:00.589550    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:00.922236 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:00.922158    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:01.198457 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.198422    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"]
Jul 28 18:43:01.199088 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.199067    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy"
Jul 28 18:43:01.199188 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.199091    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy"
Jul 28 18:43:01.199232 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.199204    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy"
Jul 28 18:43:01.199232 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.199215    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy"
Jul 28 18:43:01.201942 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.201870    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.204665 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.204646    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f36101-openshift-default-dockercfg-47z6l\""
Jul 28 18:43:01.213314 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.213290    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"]
Jul 28 18:43:01.218247 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.218224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:01.372923 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.372889    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-podinfo\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.373082 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.372948    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-envoy\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.373082 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.372982    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-certs\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.373082 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.373010    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-data\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.373082 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.373025    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-token\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.373220 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.373097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96654\" (UniqueName: \"kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-kube-api-access-96654\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.373220 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.373138    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-socket\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.373220 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.373182    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istiod-ca-cert\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.373220 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.373215    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-credential-socket\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.474492 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.474404    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-data\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.474492 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.474451    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-token\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.474492 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.474488    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96654\" (UniqueName: \"kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-kube-api-access-96654\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.474766 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.474550    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-socket\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.474766 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.474639    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istiod-ca-cert\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.475146 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.475119    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-data\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.475404 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.475372    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-socket\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.475988 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.475961    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istiod-ca-cert\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.480953 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.480924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-credential-socket\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.481068 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.480991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-podinfo\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.481152 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.481062    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-envoy\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.481152 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.481101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-certs\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.481497 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.481470    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-certs\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.483757 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.483729    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-credential-socket\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.484075 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.484055    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-token\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.484358 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.484336    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96654\" (UniqueName: \"kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-kube-api-access-96654\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.484658 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.484641    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-envoy\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.485069 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.485049    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-podinfo\") pod \"e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") " pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.512916 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.512891    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:01.623079 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.623040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:01.648169 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.648144    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"]
Jul 28 18:43:01.649749 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:43:01.649722    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c74e277_d844_43ed_8ab9_fcdd2b57376f.slice/crio-897e5b2dcab95522115f86f9029d018f5357d467a682f0564d10214f4bed800c WatchSource:0}: Error finding container 897e5b2dcab95522115f86f9029d018f5357d467a682f0564d10214f4bed800c: Status 404 returned error can't find the container with id 897e5b2dcab95522115f86f9029d018f5357d467a682f0564d10214f4bed800c
Jul 28 18:43:01.652013 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.651980    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:43:01.652105 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.652041    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:43:01.652105 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.652065    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:43:01.821919 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.821811    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" event={"ID":"2c74e277-d844-43ed-8ab9-fcdd2b57376f","Type":"ContainerStarted","Data":"2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54"}
Jul 28 18:43:01.821919 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.821891    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" event={"ID":"2c74e277-d844-43ed-8ab9-fcdd2b57376f","Type":"ContainerStarted","Data":"897e5b2dcab95522115f86f9029d018f5357d467a682f0564d10214f4bed800c"}
Jul 28 18:43:01.843194 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:01.843140    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" podStartSLOduration=0.843124262 podStartE2EDuration="843.124262ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:43:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:43:01.842365264 +0000 UTC m=+2532.918927262" watchObservedRunningTime="2026-07-28 18:43:01.843124262 +0000 UTC m=+2532.919686259"
Jul 28 18:43:02.513581 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:02.513545    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:02.518473 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:02.518449    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:02.827114 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:02.827030    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:02.827905 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:02.827887    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:03.865335 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:03.865304    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:03.928991 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:03.928956    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:03.987309 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:03.987278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:04.087655 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:04.087623    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:04.426878 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:04.426850    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:04.523647 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:04.523611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:10.992642 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:10.992605    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:43:18.692181 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:18.691698    2583 patch_prober.go:28] interesting pod/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:39140->10.132.0.56:15021: read: connection reset by peer" start-of-body=
Jul 28 18:43:18.692181 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:18.691762    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:39140->10.132.0.56:15021: read: connection reset by peer"
Jul 28 18:43:18.910566 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:18.910481    2583 generic.go:356] "Generic (PLEG): container finished" podID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerID="2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54" exitCode=0
Jul 28 18:43:18.910566 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:18.910525    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" event={"ID":"2c74e277-d844-43ed-8ab9-fcdd2b57376f","Type":"ContainerDied","Data":"2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54"}
Jul 28 18:43:18.910965 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:18.910950    2583 scope.go:117] "RemoveContainer" containerID="2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54"
Jul 28 18:43:18.912067 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:18.912040    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:43:18.912136 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:18.912104    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:43:18.912177 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:18.912134    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:43:19.917419 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:19.917387    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" event={"ID":"2c74e277-d844-43ed-8ab9-fcdd2b57376f","Type":"ContainerStarted","Data":"bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150"}
Jul 28 18:43:20.513369 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:20.513335    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:20.513590 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:20.513564    2583 patch_prober.go:28] interesting pod/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 28 18:43:20.513656 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:20.513625    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 28 18:43:21.513982 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:21.513936    2583 patch_prober.go:28] interesting pod/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 28 18:43:21.514402 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:21.513998    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 28 18:43:22.513835 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:22.513784    2583 patch_prober.go:28] interesting pod/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 28 18:43:22.514015 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:22.513856    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 28 18:43:23.513915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:23.513864    2583 patch_prober.go:28] interesting pod/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 28 18:43:23.514129 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:23.513939    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 28 18:43:24.516900 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:24.516870    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:24.517365 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:24.517305    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:24.517857 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:24.517840    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:43:28.869561 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:28.869526    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:34.222043 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:34.222006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:37.390137 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:37.390093    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:43:57.707538 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.707502    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"]
Jul 28 18:43:57.708031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.708008    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy"
Jul 28 18:43:57.708031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.708023    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9acba892-ca8b-4ac0-a645-47dab29ca9fc" containerName="istio-proxy"
Jul 28 18:43:57.709972 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.709954    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.714168 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.714142    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f36101\"/\"default-dockercfg-kskpl\""
Jul 28 18:43:57.714314 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.714177    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f36101\"/\"auth-test-model-f36101-kserve-self-signed-certs\""
Jul 28 18:43:57.714314 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.714215    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f36101\"/\"kube-root-ca.crt\""
Jul 28 18:43:57.714314 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.714157    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f36101\"/\"openshift-service-ca.crt\""
Jul 28 18:43:57.720078 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.720054    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"]
Jul 28 18:43:57.730571 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.730543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:43:57.831734 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.831696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22497911-d871-4bda-8df0-b3e61754e0b8-tls-certs\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.831968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.831752    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-model-cache\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.831968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.831895    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-tmp-dir\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.832107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.831969    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-dshm\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.832107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.832041    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99dd4\" (UniqueName: \"kubernetes.io/projected/22497911-d871-4bda-8df0-b3e61754e0b8-kube-api-access-99dd4\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.832107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.832089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-home\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.933602 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.933557    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99dd4\" (UniqueName: \"kubernetes.io/projected/22497911-d871-4bda-8df0-b3e61754e0b8-kube-api-access-99dd4\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.933781 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.933615    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-home\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.933781 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.933654    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22497911-d871-4bda-8df0-b3e61754e0b8-tls-certs\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.933781 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.933695    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-model-cache\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.933781 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.933745    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-tmp-dir\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.934045 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.933794    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-dshm\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.934130 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.934105    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-home\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.934188 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.934165    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-model-cache\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.934235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.934204    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-tmp-dir\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.936100 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.936070    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-dshm\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.936411 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.936391    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22497911-d871-4bda-8df0-b3e61754e0b8-tls-certs\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:57.942346 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:57.942321    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99dd4\" (UniqueName: \"kubernetes.io/projected/22497911-d871-4bda-8df0-b3e61754e0b8-kube-api-access-99dd4\") pod \"auth-test-model-f36101-kserve-6b7b59859-wd2cq\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") " pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:58.020348 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:58.020233    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:43:58.159638 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:58.159609    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"]
Jul 28 18:43:58.161456 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:43:58.161426    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22497911_d871_4bda_8df0_b3e61754e0b8.slice/crio-4504b8f8a05a87ac9d3c89f0072e2c1b335537dd55254e2f9d6f635ce58c93c4 WatchSource:0}: Error finding container 4504b8f8a05a87ac9d3c89f0072e2c1b335537dd55254e2f9d6f635ce58c93c4: Status 404 returned error can't find the container with id 4504b8f8a05a87ac9d3c89f0072e2c1b335537dd55254e2f9d6f635ce58c93c4
Jul 28 18:43:59.124548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:59.124508    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq" event={"ID":"22497911-d871-4bda-8df0-b3e61754e0b8","Type":"ContainerStarted","Data":"6a5b4939be2e76d05a211de6f3a609566aac457b0352ac07e0e5a0b8b1ebb928"}
Jul 28 18:43:59.124548 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:59.124551    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq" event={"ID":"22497911-d871-4bda-8df0-b3e61754e0b8","Type":"ContainerStarted","Data":"4504b8f8a05a87ac9d3c89f0072e2c1b335537dd55254e2f9d6f635ce58c93c4"}
Jul 28 18:43:59.141670 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:43:59.141618    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq" podStartSLOduration=2.141603131 podStartE2EDuration="2.141603131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:43:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:43:59.140889252 +0000 UTC m=+2590.217451250" watchObservedRunningTime="2026-07-28 18:43:59.141603131 +0000 UTC m=+2590.218165128"
Jul 28 18:44:01.586838 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:44:01.586780    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:44:08.020968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:44:08.020923    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:44:08.021394 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:44:08.021003    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:44:08.035782 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:44:08.035748    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:44:08.181043 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:44:08.181011    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:44:24.791266 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:44:24.791185    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:44:45.693259 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:44:45.693224    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:45:02.195409 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:45:02.195368    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:45:24.253424 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:45:24.253392    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:45:24.794080 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:45:24.794038    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:45:35.558132 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:45:35.558093    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:45:35.630703 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:45:35.630670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:45:41.987016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:45:41.986933    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:45:49.548668 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:45:49.548640    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:45:49.553320 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:45:49.553299    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:46:05.894372 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:46:05.894338    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:46:23.688395 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:46:23.688353    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:46:48.891025 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:46:48.890985    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:47:07.091487 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:47:07.091448    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:47:14.803473 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:47:14.803387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:47:27.587811 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:47:27.587777    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:47:30.721582 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:47:30.721533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:47:30.822022 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:47:30.821957    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:47:46.190326 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:47:46.190290    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:47:53.018719 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:47:53.018687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:48:05.788372 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:05.788332    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:48:11.446497 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:11.446457    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:48:12.292115 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:12.292079    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"]
Jul 28 18:48:12.313437 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:12.313411    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:48:19.791910 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.791879    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"]
Jul 28 18:48:19.794761 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.794746    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:19.799127 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.799104    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-37e11d\"/\"default-dockercfg-9tw8g\""
Jul 28 18:48:19.799258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.799121    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-37e11d\"/\"openshift-service-ca.crt\""
Jul 28 18:48:19.799258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.799145    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-37e11d\"/\"sub-test-model-37e11d-kserve-self-signed-certs\""
Jul 28 18:48:19.799258 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.799129    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-37e11d\"/\"kube-root-ca.crt\""
Jul 28 18:48:19.805960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.805937    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"]
Jul 28 18:48:19.814084 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.814057    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:48:19.914273 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.914239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-tmp-dir\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:19.914273 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.914273    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-dshm\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:19.914482 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.914390    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28976105-8975-4993-aa79-b80b0e117ff0-tls-certs\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:19.914520 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.914482    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w422q\" (UniqueName: \"kubernetes.io/projected/28976105-8975-4993-aa79-b80b0e117ff0-kube-api-access-w422q\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:19.914520 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.914514    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-home\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:19.914587 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:19.914530    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-model-cache\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.015566 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.015531    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w422q\" (UniqueName: \"kubernetes.io/projected/28976105-8975-4993-aa79-b80b0e117ff0-kube-api-access-w422q\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.015740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.015575    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-home\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.015740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.015594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-model-cache\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.015740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.015619    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-tmp-dir\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.015740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.015635    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-dshm\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.015740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.015700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28976105-8975-4993-aa79-b80b0e117ff0-tls-certs\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.016059 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.016039    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-home\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.016107 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.016064    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-model-cache\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.016150 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.016104    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-tmp-dir\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.018084 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.018056    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-dshm\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.018239 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.018209    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28976105-8975-4993-aa79-b80b0e117ff0-tls-certs\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.024176 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.024155    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w422q\" (UniqueName: \"kubernetes.io/projected/28976105-8975-4993-aa79-b80b0e117ff0-kube-api-access-w422q\") pod \"sub-test-model-37e11d-kserve-76dfff4d97-xl5l9\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") " pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.104450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.104364    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:20.254028 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.253987    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"]
Jul 28 18:48:20.254896 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:48:20.254866    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28976105_8975_4993_aa79_b80b0e117ff0.slice/crio-f63a1519cd40383cf62cfbc7b3eb2cf7624fdeba740418c9ce29f8ed98d28646 WatchSource:0}: Error finding container f63a1519cd40383cf62cfbc7b3eb2cf7624fdeba740418c9ce29f8ed98d28646: Status 404 returned error can't find the container with id f63a1519cd40383cf62cfbc7b3eb2cf7624fdeba740418c9ce29f8ed98d28646
Jul 28 18:48:20.441263 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.441226    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9" event={"ID":"28976105-8975-4993-aa79-b80b0e117ff0","Type":"ContainerStarted","Data":"4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b"}
Jul 28 18:48:20.441458 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.441278    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9" event={"ID":"28976105-8975-4993-aa79-b80b0e117ff0","Type":"ContainerStarted","Data":"f63a1519cd40383cf62cfbc7b3eb2cf7624fdeba740418c9ce29f8ed98d28646"}
Jul 28 18:48:20.460545 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:20.460499    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9" podStartSLOduration=1.460483657 podStartE2EDuration="1.460483657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:48:20.459337216 +0000 UTC m=+2851.535899214" watchObservedRunningTime="2026-07-28 18:48:20.460483657 +0000 UTC m=+2851.537045653"
Jul 28 18:48:22.990680 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:22.990646    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:48:27.311378 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.311325    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq" podUID="22497911-d871-4bda-8df0-b3e61754e0b8" containerName="main" containerID="cri-o://6a5b4939be2e76d05a211de6f3a609566aac457b0352ac07e0e5a0b8b1ebb928" gracePeriod=45
Jul 28 18:48:27.479951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.479791    2583 generic.go:356] "Generic (PLEG): container finished" podID="22497911-d871-4bda-8df0-b3e61754e0b8" containerID="6a5b4939be2e76d05a211de6f3a609566aac457b0352ac07e0e5a0b8b1ebb928" exitCode=0
Jul 28 18:48:27.479951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.479858    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq" event={"ID":"22497911-d871-4bda-8df0-b3e61754e0b8","Type":"ContainerDied","Data":"6a5b4939be2e76d05a211de6f3a609566aac457b0352ac07e0e5a0b8b1ebb928"}
Jul 28 18:48:27.566586 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.566526    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:48:27.694271 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694238    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-dshm\") pod \"22497911-d871-4bda-8df0-b3e61754e0b8\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") "
Jul 28 18:48:27.694453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694286    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22497911-d871-4bda-8df0-b3e61754e0b8-tls-certs\") pod \"22497911-d871-4bda-8df0-b3e61754e0b8\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") "
Jul 28 18:48:27.694453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694320    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-tmp-dir\") pod \"22497911-d871-4bda-8df0-b3e61754e0b8\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") "
Jul 28 18:48:27.694453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694387    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-home\") pod \"22497911-d871-4bda-8df0-b3e61754e0b8\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") "
Jul 28 18:48:27.694453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694446    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99dd4\" (UniqueName: \"kubernetes.io/projected/22497911-d871-4bda-8df0-b3e61754e0b8-kube-api-access-99dd4\") pod \"22497911-d871-4bda-8df0-b3e61754e0b8\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") "
Jul 28 18:48:27.694677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694498    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-model-cache\") pod \"22497911-d871-4bda-8df0-b3e61754e0b8\" (UID: \"22497911-d871-4bda-8df0-b3e61754e0b8\") "
Jul 28 18:48:27.694677 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694632    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "22497911-d871-4bda-8df0-b3e61754e0b8" (UID: "22497911-d871-4bda-8df0-b3e61754e0b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:48:27.694770 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694711    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-home" (OuterVolumeSpecName: "home") pod "22497911-d871-4bda-8df0-b3e61754e0b8" (UID: "22497911-d871-4bda-8df0-b3e61754e0b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:48:27.694847 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694766    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-model-cache" (OuterVolumeSpecName: "model-cache") pod "22497911-d871-4bda-8df0-b3e61754e0b8" (UID: "22497911-d871-4bda-8df0-b3e61754e0b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:48:27.694920 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694898    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-tmp-dir\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:48:27.694979 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694926    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-home\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:48:27.694979 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.694940    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-model-cache\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:48:27.696610 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.696585    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-dshm" (OuterVolumeSpecName: "dshm") pod "22497911-d871-4bda-8df0-b3e61754e0b8" (UID: "22497911-d871-4bda-8df0-b3e61754e0b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:48:27.696724 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.696620    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22497911-d871-4bda-8df0-b3e61754e0b8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "22497911-d871-4bda-8df0-b3e61754e0b8" (UID: "22497911-d871-4bda-8df0-b3e61754e0b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:48:27.696724 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.696669    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22497911-d871-4bda-8df0-b3e61754e0b8-kube-api-access-99dd4" (OuterVolumeSpecName: "kube-api-access-99dd4") pod "22497911-d871-4bda-8df0-b3e61754e0b8" (UID: "22497911-d871-4bda-8df0-b3e61754e0b8"). InnerVolumeSpecName "kube-api-access-99dd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:48:27.795920 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.795884    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99dd4\" (UniqueName: \"kubernetes.io/projected/22497911-d871-4bda-8df0-b3e61754e0b8-kube-api-access-99dd4\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:48:27.795920 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.795915    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22497911-d871-4bda-8df0-b3e61754e0b8-dshm\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:48:27.795920 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:27.795924    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22497911-d871-4bda-8df0-b3e61754e0b8-tls-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:48:28.485747 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:28.485713    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"
Jul 28 18:48:28.485747 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:28.485746    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq" event={"ID":"22497911-d871-4bda-8df0-b3e61754e0b8","Type":"ContainerDied","Data":"4504b8f8a05a87ac9d3c89f0072e2c1b335537dd55254e2f9d6f635ce58c93c4"}
Jul 28 18:48:28.486301 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:28.485795    2583 scope.go:117] "RemoveContainer" containerID="6a5b4939be2e76d05a211de6f3a609566aac457b0352ac07e0e5a0b8b1ebb928"
Jul 28 18:48:28.509464 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:28.509435    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"]
Jul 28 18:48:28.511226 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:28.511205    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f36101/auth-test-model-f36101-kserve-6b7b59859-wd2cq"]
Jul 28 18:48:29.405987 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:29.405949    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22497911-d871-4bda-8df0-b3e61754e0b8" path="/var/lib/kubelet/pods/22497911-d871-4bda-8df0-b3e61754e0b8/volumes"
Jul 28 18:48:30.104615 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:30.104579    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:30.105031 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:30.104632    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:30.117200 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:30.117175    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:30.514645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:30.514612    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:48:43.039247 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:43.039168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:48:46.683179 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:48:46.683140    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:49:20.391873 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:49:20.391833    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:49:38.592235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:49:38.592203    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:49:51.136218 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:49:51.136180    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"]
Jul 28 18:49:51.154909 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:49:51.154881    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:49:52.104188 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:49:52.104150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:49:53.488969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:49:53.488935    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:49:56.394614 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:49:56.394570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:50:06.155646 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.155583    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9" podUID="28976105-8975-4993-aa79-b80b0e117ff0" containerName="main" containerID="cri-o://4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b" gracePeriod=45
Jul 28 18:50:06.404490 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.404464    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:50:06.533807 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.533771    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-dshm\") pod \"28976105-8975-4993-aa79-b80b0e117ff0\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") "
Jul 28 18:50:06.534036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.533858    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w422q\" (UniqueName: \"kubernetes.io/projected/28976105-8975-4993-aa79-b80b0e117ff0-kube-api-access-w422q\") pod \"28976105-8975-4993-aa79-b80b0e117ff0\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") "
Jul 28 18:50:06.534036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.533918    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-tmp-dir\") pod \"28976105-8975-4993-aa79-b80b0e117ff0\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") "
Jul 28 18:50:06.534036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.533936    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28976105-8975-4993-aa79-b80b0e117ff0-tls-certs\") pod \"28976105-8975-4993-aa79-b80b0e117ff0\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") "
Jul 28 18:50:06.534221 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.534059    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-home\") pod \"28976105-8975-4993-aa79-b80b0e117ff0\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") "
Jul 28 18:50:06.534221 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.534140    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "28976105-8975-4993-aa79-b80b0e117ff0" (UID: "28976105-8975-4993-aa79-b80b0e117ff0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:50:06.534353 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.534319    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-home" (OuterVolumeSpecName: "home") pod "28976105-8975-4993-aa79-b80b0e117ff0" (UID: "28976105-8975-4993-aa79-b80b0e117ff0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:50:06.534412 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.534368    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-model-cache\") pod \"28976105-8975-4993-aa79-b80b0e117ff0\" (UID: \"28976105-8975-4993-aa79-b80b0e117ff0\") "
Jul 28 18:50:06.534477 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.534449    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-model-cache" (OuterVolumeSpecName: "model-cache") pod "28976105-8975-4993-aa79-b80b0e117ff0" (UID: "28976105-8975-4993-aa79-b80b0e117ff0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:50:06.534801 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.534780    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-model-cache\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:50:06.534934 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.534808    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-tmp-dir\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:50:06.534934 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.534861    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-home\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:50:06.536309 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.536269    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28976105-8975-4993-aa79-b80b0e117ff0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "28976105-8975-4993-aa79-b80b0e117ff0" (UID: "28976105-8975-4993-aa79-b80b0e117ff0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:50:06.536437 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.536333    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-dshm" (OuterVolumeSpecName: "dshm") pod "28976105-8975-4993-aa79-b80b0e117ff0" (UID: "28976105-8975-4993-aa79-b80b0e117ff0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:50:06.536636 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.536605    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28976105-8975-4993-aa79-b80b0e117ff0-kube-api-access-w422q" (OuterVolumeSpecName: "kube-api-access-w422q") pod "28976105-8975-4993-aa79-b80b0e117ff0" (UID: "28976105-8975-4993-aa79-b80b0e117ff0"). InnerVolumeSpecName "kube-api-access-w422q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:50:06.591691 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.591658    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:50:06.636578 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.636539    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28976105-8975-4993-aa79-b80b0e117ff0-tls-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:50:06.636578 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.636575    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28976105-8975-4993-aa79-b80b0e117ff0-dshm\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:50:06.636783 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.636590    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w422q\" (UniqueName: \"kubernetes.io/projected/28976105-8975-4993-aa79-b80b0e117ff0-kube-api-access-w422q\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:50:06.985467 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.985432    2583 generic.go:356] "Generic (PLEG): container finished" podID="28976105-8975-4993-aa79-b80b0e117ff0" containerID="4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b" exitCode=0
Jul 28 18:50:06.985661 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.985480    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"
Jul 28 18:50:06.985661 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.985526    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9" event={"ID":"28976105-8975-4993-aa79-b80b0e117ff0","Type":"ContainerDied","Data":"4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b"}
Jul 28 18:50:06.985661 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.985573    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9" event={"ID":"28976105-8975-4993-aa79-b80b0e117ff0","Type":"ContainerDied","Data":"f63a1519cd40383cf62cfbc7b3eb2cf7624fdeba740418c9ce29f8ed98d28646"}
Jul 28 18:50:06.985661 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.985591    2583 scope.go:117] "RemoveContainer" containerID="4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b"
Jul 28 18:50:06.994899 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.994880    2583 scope.go:117] "RemoveContainer" containerID="4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b"
Jul 28 18:50:06.995127 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:50:06.995111    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b\": container with ID starting with 4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b not found: ID does not exist" containerID="4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b"
Jul 28 18:50:06.995184 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:06.995135    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b"} err="failed to get container status \"4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b\": rpc error: code = NotFound desc = could not find container \"4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b\": container with ID starting with 4453180187a8640d4d3ffa40e5884b618e55f85a017ff7ef2d2edd76c1cb4c8b not found: ID does not exist"
Jul 28 18:50:07.006920 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:07.006895    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"]
Jul 28 18:50:07.010947 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:07.010923    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-37e11d/sub-test-model-37e11d-kserve-76dfff4d97-xl5l9"]
Jul 28 18:50:07.406096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:07.406013    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28976105-8975-4993-aa79-b80b0e117ff0" path="/var/lib/kubelet/pods/28976105-8975-4993-aa79-b80b0e117ff0/volumes"
Jul 28 18:50:25.312991 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.312959    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"]
Jul 28 18:50:25.313467 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.313340    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22497911-d871-4bda-8df0-b3e61754e0b8" containerName="main"
Jul 28 18:50:25.313467 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.313353    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="22497911-d871-4bda-8df0-b3e61754e0b8" containerName="main"
Jul 28 18:50:25.313467 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.313387    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28976105-8975-4993-aa79-b80b0e117ff0" containerName="main"
Jul 28 18:50:25.313467 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.313396    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="28976105-8975-4993-aa79-b80b0e117ff0" containerName="main"
Jul 28 18:50:25.313630 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.313485    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="22497911-d871-4bda-8df0-b3e61754e0b8" containerName="main"
Jul 28 18:50:25.313630 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.313499    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="28976105-8975-4993-aa79-b80b0e117ff0" containerName="main"
Jul 28 18:50:25.318073 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.318051    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.322324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.322298    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f36101\"/\"rate-test-model-f36101-kserve-self-signed-certs\""
Jul 28 18:50:25.322440 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.322300    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f36101\"/\"default-dockercfg-kskpl\""
Jul 28 18:50:25.322440 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.322300    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f36101\"/\"openshift-service-ca.crt\""
Jul 28 18:50:25.322440 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.322355    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f36101\"/\"kube-root-ca.crt\""
Jul 28 18:50:25.328012 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.327990    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"]
Jul 28 18:50:25.330547 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.330510    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:50:25.422363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.422329    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-model-cache\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.422529 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.422376    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-home\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.422529 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.422499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d31a90-c810-4a93-8e23-e0f779223f29-tls-certs\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.422615 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.422528    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwljz\" (UniqueName: \"kubernetes.io/projected/b4d31a90-c810-4a93-8e23-e0f779223f29-kube-api-access-dwljz\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.422615 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.422555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-tmp-dir\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.422755 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.422656    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-dshm\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.523495 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.523466    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwljz\" (UniqueName: \"kubernetes.io/projected/b4d31a90-c810-4a93-8e23-e0f779223f29-kube-api-access-dwljz\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.523684 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.523500    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-tmp-dir\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.523684 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.523539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-dshm\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.523684 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.523572    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-model-cache\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.523684 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.523594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-home\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.523684 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.523653    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d31a90-c810-4a93-8e23-e0f779223f29-tls-certs\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.523979 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.523954    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-tmp-dir\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.524045 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.523978    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-model-cache\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.524138 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.524118    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-home\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.525795 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.525773    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-dshm\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.526071 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.526052    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d31a90-c810-4a93-8e23-e0f779223f29-tls-certs\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.533337 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.533314    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwljz\" (UniqueName: \"kubernetes.io/projected/b4d31a90-c810-4a93-8e23-e0f779223f29-kube-api-access-dwljz\") pod \"rate-test-model-f36101-kserve-865d6cdc6c-4tmng\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") " pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.629350 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.629259    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:25.771624 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:25.771598    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"]
Jul 28 18:50:25.773533 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:50:25.773508    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4d31a90_c810_4a93_8e23_e0f779223f29.slice/crio-d6b703d65f06a5e694648be1527106fe3bdd48b93c379396d9173af28de865c1 WatchSource:0}: Error finding container d6b703d65f06a5e694648be1527106fe3bdd48b93c379396d9173af28de865c1: Status 404 returned error can't find the container with id d6b703d65f06a5e694648be1527106fe3bdd48b93c379396d9173af28de865c1
Jul 28 18:50:26.090406 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:26.090362    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng" event={"ID":"b4d31a90-c810-4a93-8e23-e0f779223f29","Type":"ContainerStarted","Data":"8eea14c5664cafafd4c57d6d16a7dcc37275102006cde2af8ed10271efe7c965"}
Jul 28 18:50:26.090406 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:26.090410    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng" event={"ID":"b4d31a90-c810-4a93-8e23-e0f779223f29","Type":"ContainerStarted","Data":"d6b703d65f06a5e694648be1527106fe3bdd48b93c379396d9173af28de865c1"}
Jul 28 18:50:26.109623 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:26.109566    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng" podStartSLOduration=1.109551391 podStartE2EDuration="1.109551391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:50:26.108045029 +0000 UTC m=+2977.184607027" watchObservedRunningTime="2026-07-28 18:50:26.109551391 +0000 UTC m=+2977.186113387"
Jul 28 18:50:29.096106 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:29.096066    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:50:35.629980 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:35.629940    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:35.630379 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:35.630003    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:35.642462 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:35.642430    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:36.151791 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:36.151758    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:50:47.884969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:47.884934    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:50:49.579850 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:49.579801    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:50:49.583740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:49.583708    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:50:54.630727 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:50:54.630690    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:00.588443 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:00.588400    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:51:21.994969 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:21.994935    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:51:28.455868 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:28.455831    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:29.303958 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:29.303925    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"]
Jul 28 18:51:29.321525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:29.321481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:30.372308 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:30.372270    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7351-140545011633328"]
Jul 28 18:51:30.375587 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:30.375568    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545011633328"
Jul 28 18:51:30.378317 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:30.378294    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-swrnk\""
Jul 28 18:51:30.382954 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:30.382808    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140545011633328"]
Jul 28 18:51:30.563452 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:30.563415    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpzxn\" (UniqueName: \"kubernetes.io/projected/fc5d394c-d2d8-406c-9851-0eebe8df9be9-kube-api-access-lpzxn\") pod \"test-curl-7351-140545011633328\" (UID: \"fc5d394c-d2d8-406c-9851-0eebe8df9be9\") " pod="opendatahub/test-curl-7351-140545011633328"
Jul 28 18:51:30.665178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:30.665086    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpzxn\" (UniqueName: \"kubernetes.io/projected/fc5d394c-d2d8-406c-9851-0eebe8df9be9-kube-api-access-lpzxn\") pod \"test-curl-7351-140545011633328\" (UID: \"fc5d394c-d2d8-406c-9851-0eebe8df9be9\") " pod="opendatahub/test-curl-7351-140545011633328"
Jul 28 18:51:30.674300 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:30.674266    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpzxn\" (UniqueName: \"kubernetes.io/projected/fc5d394c-d2d8-406c-9851-0eebe8df9be9-kube-api-access-lpzxn\") pod \"test-curl-7351-140545011633328\" (UID: \"fc5d394c-d2d8-406c-9851-0eebe8df9be9\") " pod="opendatahub/test-curl-7351-140545011633328"
Jul 28 18:51:30.685159 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:30.685133    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545011633328"
Jul 28 18:51:30.822614 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:30.822528    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140545011633328"]
Jul 28 18:51:30.826089 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:51:30.826063    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc5d394c_d2d8_406c_9851_0eebe8df9be9.slice/crio-401faa813767bb941355be51853e76cbe8b0a7756e4b7829829da1bd66ae5d00 WatchSource:0}: Error finding container 401faa813767bb941355be51853e76cbe8b0a7756e4b7829829da1bd66ae5d00: Status 404 returned error can't find the container with id 401faa813767bb941355be51853e76cbe8b0a7756e4b7829829da1bd66ae5d00
Jul 28 18:51:30.827412 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:30.827392    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:51:31.418246 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:31.418202    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140545011633328" event={"ID":"fc5d394c-d2d8-406c-9851-0eebe8df9be9","Type":"ContainerStarted","Data":"401faa813767bb941355be51853e76cbe8b0a7756e4b7829829da1bd66ae5d00"}
Jul 28 18:51:32.424990 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:32.424954    2583 generic.go:356] "Generic (PLEG): container finished" podID="fc5d394c-d2d8-406c-9851-0eebe8df9be9" containerID="e306830dacdcfb767148e62642f983e373cf60ae4f2f89a9f935a3083559aef7" exitCode=0
Jul 28 18:51:32.425419 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:32.425041    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140545011633328" event={"ID":"fc5d394c-d2d8-406c-9851-0eebe8df9be9","Type":"ContainerDied","Data":"e306830dacdcfb767148e62642f983e373cf60ae4f2f89a9f935a3083559aef7"}
Jul 28 18:51:33.567096 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:33.567067    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545011633328"
Jul 28 18:51:33.587748 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:33.587722    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7351-140545011633328_fc5d394c-d2d8-406c-9851-0eebe8df9be9/test-curl-7351-140545011633328/0.log"
Jul 28 18:51:33.591974 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:33.591949    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpzxn\" (UniqueName: \"kubernetes.io/projected/fc5d394c-d2d8-406c-9851-0eebe8df9be9-kube-api-access-lpzxn\") pod \"fc5d394c-d2d8-406c-9851-0eebe8df9be9\" (UID: \"fc5d394c-d2d8-406c-9851-0eebe8df9be9\") "
Jul 28 18:51:33.594227 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:33.594198    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc5d394c-d2d8-406c-9851-0eebe8df9be9-kube-api-access-lpzxn" (OuterVolumeSpecName: "kube-api-access-lpzxn") pod "fc5d394c-d2d8-406c-9851-0eebe8df9be9" (UID: "fc5d394c-d2d8-406c-9851-0eebe8df9be9"). InnerVolumeSpecName "kube-api-access-lpzxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:51:33.609396 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:33.609365    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7351-140545011633328"]
Jul 28 18:51:33.613793 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:33.613767    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7351-140545011633328"]
Jul 28 18:51:33.692869 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:33.692841    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpzxn\" (UniqueName: \"kubernetes.io/projected/fc5d394c-d2d8-406c-9851-0eebe8df9be9-kube-api-access-lpzxn\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:51:34.436639 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:34.436606    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545011633328"
Jul 28 18:51:34.436791 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:34.436645    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="401faa813767bb941355be51853e76cbe8b0a7756e4b7829829da1bd66ae5d00"
Jul 28 18:51:34.447327 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:51:34.447298    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7351-140545011633328\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="fc5d394c-d2d8-406c-9851-0eebe8df9be9" pod="opendatahub/test-curl-7351-140545011633328"
Jul 28 18:51:34.936290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:34.936246    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:35.037492 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:35.037449    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:35.406061 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:35.406027    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc5d394c-d2d8-406c-9851-0eebe8df9be9" path="/var/lib/kubelet/pods/fc5d394c-d2d8-406c-9851-0eebe8df9be9/volumes"
Jul 28 18:51:36.569583 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.569549    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7351-140545006178496"]
Jul 28 18:51:36.569970 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.569950    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc5d394c-d2d8-406c-9851-0eebe8df9be9" containerName="test-curl-7351-140545011633328"
Jul 28 18:51:36.569970 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.569963    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc5d394c-d2d8-406c-9851-0eebe8df9be9" containerName="test-curl-7351-140545011633328"
Jul 28 18:51:36.570051 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.570042    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc5d394c-d2d8-406c-9851-0eebe8df9be9" containerName="test-curl-7351-140545011633328"
Jul 28 18:51:36.573164 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.573148    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545006178496"
Jul 28 18:51:36.576165 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.576142    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-swrnk\""
Jul 28 18:51:36.580621 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.580589    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140545006178496"]
Jul 28 18:51:36.622782 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.622752    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt48b\" (UniqueName: \"kubernetes.io/projected/98001fdd-7574-431c-9150-9d8fcdcadbf0-kube-api-access-pt48b\") pod \"test-curl-7351-140545006178496\" (UID: \"98001fdd-7574-431c-9150-9d8fcdcadbf0\") " pod="opendatahub/test-curl-7351-140545006178496"
Jul 28 18:51:36.724330 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.724288    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt48b\" (UniqueName: \"kubernetes.io/projected/98001fdd-7574-431c-9150-9d8fcdcadbf0-kube-api-access-pt48b\") pod \"test-curl-7351-140545006178496\" (UID: \"98001fdd-7574-431c-9150-9d8fcdcadbf0\") " pod="opendatahub/test-curl-7351-140545006178496"
Jul 28 18:51:36.733010 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.732984    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt48b\" (UniqueName: \"kubernetes.io/projected/98001fdd-7574-431c-9150-9d8fcdcadbf0-kube-api-access-pt48b\") pod \"test-curl-7351-140545006178496\" (UID: \"98001fdd-7574-431c-9150-9d8fcdcadbf0\") " pod="opendatahub/test-curl-7351-140545006178496"
Jul 28 18:51:36.884361 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:36.884268    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545006178496"
Jul 28 18:51:37.018766 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:37.018735    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140545006178496"]
Jul 28 18:51:37.020120 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:51:37.020089    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98001fdd_7574_431c_9150_9d8fcdcadbf0.slice/crio-94a2b3fd282cc8758c13f8bedcb587306504cd87f93bdc9b103cfa948a4da282 WatchSource:0}: Error finding container 94a2b3fd282cc8758c13f8bedcb587306504cd87f93bdc9b103cfa948a4da282: Status 404 returned error can't find the container with id 94a2b3fd282cc8758c13f8bedcb587306504cd87f93bdc9b103cfa948a4da282
Jul 28 18:51:37.453863 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:37.453829    2583 generic.go:356] "Generic (PLEG): container finished" podID="98001fdd-7574-431c-9150-9d8fcdcadbf0" containerID="bbdb6b516349be9faedd15fea1cabad9d53a495c93f7a5302f815d65094ab353" exitCode=0
Jul 28 18:51:37.454016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:37.453916    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140545006178496" event={"ID":"98001fdd-7574-431c-9150-9d8fcdcadbf0","Type":"ContainerDied","Data":"bbdb6b516349be9faedd15fea1cabad9d53a495c93f7a5302f815d65094ab353"}
Jul 28 18:51:37.454016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:37.453955    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140545006178496" event={"ID":"98001fdd-7574-431c-9150-9d8fcdcadbf0","Type":"ContainerStarted","Data":"94a2b3fd282cc8758c13f8bedcb587306504cd87f93bdc9b103cfa948a4da282"}
Jul 28 18:51:38.598836 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:38.598793    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545006178496"
Jul 28 18:51:38.618952 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:38.618857    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7351-140545006178496_98001fdd-7574-431c-9150-9d8fcdcadbf0/test-curl-7351-140545006178496/0.log"
Jul 28 18:51:38.639255 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:38.639228    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7351-140545006178496"]
Jul 28 18:51:38.642656 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:38.642629    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7351-140545006178496"]
Jul 28 18:51:38.642950 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:38.642930    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt48b\" (UniqueName: \"kubernetes.io/projected/98001fdd-7574-431c-9150-9d8fcdcadbf0-kube-api-access-pt48b\") pod \"98001fdd-7574-431c-9150-9d8fcdcadbf0\" (UID: \"98001fdd-7574-431c-9150-9d8fcdcadbf0\") "
Jul 28 18:51:38.645298 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:38.645273    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98001fdd-7574-431c-9150-9d8fcdcadbf0-kube-api-access-pt48b" (OuterVolumeSpecName: "kube-api-access-pt48b") pod "98001fdd-7574-431c-9150-9d8fcdcadbf0" (UID: "98001fdd-7574-431c-9150-9d8fcdcadbf0"). InnerVolumeSpecName "kube-api-access-pt48b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:51:38.743838 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:38.743742    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt48b\" (UniqueName: \"kubernetes.io/projected/98001fdd-7574-431c-9150-9d8fcdcadbf0-kube-api-access-pt48b\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:51:39.405588 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:39.405553    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98001fdd-7574-431c-9150-9d8fcdcadbf0" path="/var/lib/kubelet/pods/98001fdd-7574-431c-9150-9d8fcdcadbf0/volumes"
Jul 28 18:51:39.465154 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:39.465122    2583 scope.go:117] "RemoveContainer" containerID="bbdb6b516349be9faedd15fea1cabad9d53a495c93f7a5302f815d65094ab353"
Jul 28 18:51:39.465154 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:39.465145    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545006178496"
Jul 28 18:51:41.291731 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.291651    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:51:41.526925 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.526892    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7351-140545006179504"]
Jul 28 18:51:41.527324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.527311    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98001fdd-7574-431c-9150-9d8fcdcadbf0" containerName="test-curl-7351-140545006178496"
Jul 28 18:51:41.527380 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.527328    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="98001fdd-7574-431c-9150-9d8fcdcadbf0" containerName="test-curl-7351-140545006178496"
Jul 28 18:51:41.527417 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.527391    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="98001fdd-7574-431c-9150-9d8fcdcadbf0" containerName="test-curl-7351-140545006178496"
Jul 28 18:51:41.531944 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.531922    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545006179504"
Jul 28 18:51:41.535298 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.535234    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-swrnk\""
Jul 28 18:51:41.537660 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.537634    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140545006179504"]
Jul 28 18:51:41.572870 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.572788    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn9kx\" (UniqueName: \"kubernetes.io/projected/244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2-kube-api-access-fn9kx\") pod \"test-curl-7351-140545006179504\" (UID: \"244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2\") " pod="opendatahub/test-curl-7351-140545006179504"
Jul 28 18:51:41.674072 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.674038    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn9kx\" (UniqueName: \"kubernetes.io/projected/244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2-kube-api-access-fn9kx\") pod \"test-curl-7351-140545006179504\" (UID: \"244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2\") " pod="opendatahub/test-curl-7351-140545006179504"
Jul 28 18:51:41.682773 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.682750    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn9kx\" (UniqueName: \"kubernetes.io/projected/244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2-kube-api-access-fn9kx\") pod \"test-curl-7351-140545006179504\" (UID: \"244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2\") " pod="opendatahub/test-curl-7351-140545006179504"
Jul 28 18:51:41.844132 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.844043    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545006179504"
Jul 28 18:51:41.984425 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:41.984400    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7351-140545006179504"]
Jul 28 18:51:41.986631 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:51:41.986599    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod244bdfc9_c10b_4f13_8c50_d17c4b8d1ac2.slice/crio-17d7e3c5c9ddf35a391095c495187dffb9a5cfa7d0c71442ad10a6057cff7732 WatchSource:0}: Error finding container 17d7e3c5c9ddf35a391095c495187dffb9a5cfa7d0c71442ad10a6057cff7732: Status 404 returned error can't find the container with id 17d7e3c5c9ddf35a391095c495187dffb9a5cfa7d0c71442ad10a6057cff7732
Jul 28 18:51:42.482118 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:42.482085    2583 generic.go:356] "Generic (PLEG): container finished" podID="244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2" containerID="00a18ee65e0f31601f0860ed1fc0574c89a144b43e9de37348090187325a8340" exitCode=0
Jul 28 18:51:42.482476 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:42.482173    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140545006179504" event={"ID":"244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2","Type":"ContainerDied","Data":"00a18ee65e0f31601f0860ed1fc0574c89a144b43e9de37348090187325a8340"}
Jul 28 18:51:42.482476 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:42.482222    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7351-140545006179504" event={"ID":"244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2","Type":"ContainerStarted","Data":"17d7e3c5c9ddf35a391095c495187dffb9a5cfa7d0c71442ad10a6057cff7732"}
Jul 28 18:51:43.619984 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:43.619958    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545006179504"
Jul 28 18:51:43.639738 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:43.639694    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7351-140545006179504_244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2/test-curl-7351-140545006179504/0.log"
Jul 28 18:51:43.659565 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:43.659520    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7351-140545006179504"]
Jul 28 18:51:43.662796 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:43.662770    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7351-140545006179504"]
Jul 28 18:51:43.692209 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:43.692176    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn9kx\" (UniqueName: \"kubernetes.io/projected/244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2-kube-api-access-fn9kx\") pod \"244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2\" (UID: \"244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2\") "
Jul 28 18:51:43.694232 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:43.694204    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2-kube-api-access-fn9kx" (OuterVolumeSpecName: "kube-api-access-fn9kx") pod "244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2" (UID: "244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2"). InnerVolumeSpecName "kube-api-access-fn9kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:51:43.793607 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:43.793528    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn9kx\" (UniqueName: \"kubernetes.io/projected/244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2-kube-api-access-fn9kx\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:51:44.323414 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.323367    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng" podUID="b4d31a90-c810-4a93-8e23-e0f779223f29" containerName="main" containerID="cri-o://8eea14c5664cafafd4c57d6d16a7dcc37275102006cde2af8ed10271efe7c965" gracePeriod=45
Jul 28 18:51:44.494952 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.494922    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7351-140545006179504"
Jul 28 18:51:44.494952 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.494951    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17d7e3c5c9ddf35a391095c495187dffb9a5cfa7d0c71442ad10a6057cff7732"
Jul 28 18:51:44.497376 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.497344    2583 generic.go:356] "Generic (PLEG): container finished" podID="b4d31a90-c810-4a93-8e23-e0f779223f29" containerID="8eea14c5664cafafd4c57d6d16a7dcc37275102006cde2af8ed10271efe7c965" exitCode=0
Jul 28 18:51:44.497499 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.497436    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng" event={"ID":"b4d31a90-c810-4a93-8e23-e0f779223f29","Type":"ContainerDied","Data":"8eea14c5664cafafd4c57d6d16a7dcc37275102006cde2af8ed10271efe7c965"}
Jul 28 18:51:44.506695 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:51:44.506665    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7351-140545006179504\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2" pod="opendatahub/test-curl-7351-140545006179504"
Jul 28 18:51:44.574202 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.574142    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:51:44.594638 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:51:44.594594    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7351-140545006179504\" is forbidden: User \"system:node:ip-10-0-141-192.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-192.ec2.internal' and this object" podUID="244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2" pod="opendatahub/test-curl-7351-140545006179504"
Jul 28 18:51:44.701710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.701673    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwljz\" (UniqueName: \"kubernetes.io/projected/b4d31a90-c810-4a93-8e23-e0f779223f29-kube-api-access-dwljz\") pod \"b4d31a90-c810-4a93-8e23-e0f779223f29\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") "
Jul 28 18:51:44.701710 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.701716    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-dshm\") pod \"b4d31a90-c810-4a93-8e23-e0f779223f29\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") "
Jul 28 18:51:44.702238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.701745    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-home\") pod \"b4d31a90-c810-4a93-8e23-e0f779223f29\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") "
Jul 28 18:51:44.702238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.701785    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-model-cache\") pod \"b4d31a90-c810-4a93-8e23-e0f779223f29\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") "
Jul 28 18:51:44.702238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.701876    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-tmp-dir\") pod \"b4d31a90-c810-4a93-8e23-e0f779223f29\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") "
Jul 28 18:51:44.702238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.701959    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d31a90-c810-4a93-8e23-e0f779223f29-tls-certs\") pod \"b4d31a90-c810-4a93-8e23-e0f779223f29\" (UID: \"b4d31a90-c810-4a93-8e23-e0f779223f29\") "
Jul 28 18:51:44.702238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.702015    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-home" (OuterVolumeSpecName: "home") pod "b4d31a90-c810-4a93-8e23-e0f779223f29" (UID: "b4d31a90-c810-4a93-8e23-e0f779223f29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:44.702238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.702137    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-model-cache" (OuterVolumeSpecName: "model-cache") pod "b4d31a90-c810-4a93-8e23-e0f779223f29" (UID: "b4d31a90-c810-4a93-8e23-e0f779223f29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:44.702238 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.702172    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b4d31a90-c810-4a93-8e23-e0f779223f29" (UID: "b4d31a90-c810-4a93-8e23-e0f779223f29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:44.702479 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.702273    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-home\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:51:44.702479 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.702286    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-model-cache\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:51:44.702479 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.702301    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-tmp-dir\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:51:44.704015 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.703988    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-dshm" (OuterVolumeSpecName: "dshm") pod "b4d31a90-c810-4a93-8e23-e0f779223f29" (UID: "b4d31a90-c810-4a93-8e23-e0f779223f29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:44.704015 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.703994    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4d31a90-c810-4a93-8e23-e0f779223f29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4d31a90-c810-4a93-8e23-e0f779223f29" (UID: "b4d31a90-c810-4a93-8e23-e0f779223f29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:51:44.704015 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.703987    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4d31a90-c810-4a93-8e23-e0f779223f29-kube-api-access-dwljz" (OuterVolumeSpecName: "kube-api-access-dwljz") pod "b4d31a90-c810-4a93-8e23-e0f779223f29" (UID: "b4d31a90-c810-4a93-8e23-e0f779223f29"). InnerVolumeSpecName "kube-api-access-dwljz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:51:44.803134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.803081    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d31a90-c810-4a93-8e23-e0f779223f29-tls-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:51:44.803134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.803128    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwljz\" (UniqueName: \"kubernetes.io/projected/b4d31a90-c810-4a93-8e23-e0f779223f29-kube-api-access-dwljz\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:51:44.803134 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:44.803138    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d31a90-c810-4a93-8e23-e0f779223f29-dshm\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:51:45.412123 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:45.412089    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2" path="/var/lib/kubelet/pods/244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2/volumes"
Jul 28 18:51:45.503742 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:45.503711    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng" event={"ID":"b4d31a90-c810-4a93-8e23-e0f779223f29","Type":"ContainerDied","Data":"d6b703d65f06a5e694648be1527106fe3bdd48b93c379396d9173af28de865c1"}
Jul 28 18:51:45.503915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:45.503764    2583 scope.go:117] "RemoveContainer" containerID="8eea14c5664cafafd4c57d6d16a7dcc37275102006cde2af8ed10271efe7c965"
Jul 28 18:51:45.503915 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:45.503719    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"
Jul 28 18:51:45.522937 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:45.522908    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"]
Jul 28 18:51:45.527058 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:45.527034    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f36101/rate-test-model-f36101-kserve-865d6cdc6c-4tmng"]
Jul 28 18:51:46.939904 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:46.939870    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:47.296719 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.296642    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"]
Jul 28 18:51:47.297069 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.297049    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d31a90-c810-4a93-8e23-e0f779223f29" containerName="main"
Jul 28 18:51:47.297069 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.297066    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d31a90-c810-4a93-8e23-e0f779223f29" containerName="main"
Jul 28 18:51:47.297069 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.297091    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2" containerName="test-curl-7351-140545006179504"
Jul 28 18:51:47.297069 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.297097    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2" containerName="test-curl-7351-140545006179504"
Jul 28 18:51:47.297352 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.297206    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="244bdfc9-c10b-4f13-8c50-d17c4b8d1ac2" containerName="test-curl-7351-140545006179504"
Jul 28 18:51:47.297352 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.297224    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4d31a90-c810-4a93-8e23-e0f779223f29" containerName="main"
Jul 28 18:51:47.301739 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.301715    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.304650 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.304631    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-c1237e-openshift-default-dockercfg-9ncbx\""
Jul 28 18:51:47.312911 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.312880    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"]
Jul 28 18:51:47.314110 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.314082    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:47.408296 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.408260    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4d31a90-c810-4a93-8e23-e0f779223f29" path="/var/lib/kubelet/pods/b4d31a90-c810-4a93-8e23-e0f779223f29/volumes"
Jul 28 18:51:47.430516 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.430482    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-socket\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.430712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.430561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2hsr\" (UniqueName: \"kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-kube-api-access-x2hsr\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.430712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.430594    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-credential-socket\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.430712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.430637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-data\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.430891 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.430709    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-envoy\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.430891 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.430759    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3798bf77-2864-411e-8037-56e0d3d244b1-istio-podinfo\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.430891 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.430853    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-certs\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.431014 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.430935    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-istio-token\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.431014 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.430966    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3798bf77-2864-411e-8037-56e0d3d244b1-istiod-ca-cert\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.532287 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.532239    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-envoy\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.532287 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.532292    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3798bf77-2864-411e-8037-56e0d3d244b1-istio-podinfo\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.532558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.532327    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-certs\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.532558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.532379    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-istio-token\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.532558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.532409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3798bf77-2864-411e-8037-56e0d3d244b1-istiod-ca-cert\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.532558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.532447    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-socket\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.532558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.532513    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2hsr\" (UniqueName: \"kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-kube-api-access-x2hsr\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.532558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.532544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-credential-socket\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.532883 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.532589    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-data\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.532965 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.532943    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-data\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.533476 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.533173    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-socket\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.533476 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.533314    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-certs\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.533476 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.533356    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3798bf77-2864-411e-8037-56e0d3d244b1-istiod-ca-cert\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.533712 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.533545    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-credential-socket\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.535549 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.535524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-envoy\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.535718 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.535699    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3798bf77-2864-411e-8037-56e0d3d244b1-istio-podinfo\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.542572 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.542534    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2hsr\" (UniqueName: \"kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-kube-api-access-x2hsr\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.542698 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.542605    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-istio-token\") pod \"e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") " pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.613592 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.613498    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:47.636403 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.636363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:47.773054 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.773023    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"]
Jul 28 18:51:47.774884 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:51:47.774854    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3798bf77_2864_411e_8037_56e0d3d244b1.slice/crio-a2fc1bb2d6a2b2eccb1bae21c46c6461bd0d632dc9548b14cc493298f801bb38 WatchSource:0}: Error finding container a2fc1bb2d6a2b2eccb1bae21c46c6461bd0d632dc9548b14cc493298f801bb38: Status 404 returned error can't find the container with id a2fc1bb2d6a2b2eccb1bae21c46c6461bd0d632dc9548b14cc493298f801bb38
Jul 28 18:51:47.777478 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.777448    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:51:47.777565 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.777510    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:51:47.777565 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:47.777539    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:51:48.522694 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:48.522654    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" event={"ID":"3798bf77-2864-411e-8037-56e0d3d244b1","Type":"ContainerStarted","Data":"dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd"}
Jul 28 18:51:48.523163 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:48.522717    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" event={"ID":"3798bf77-2864-411e-8037-56e0d3d244b1","Type":"ContainerStarted","Data":"a2fc1bb2d6a2b2eccb1bae21c46c6461bd0d632dc9548b14cc493298f801bb38"}
Jul 28 18:51:48.546288 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:48.546231    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" podStartSLOduration=1.546209789 podStartE2EDuration="1.546209789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:51:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:51:48.544433417 +0000 UTC m=+3059.620995414" watchObservedRunningTime="2026-07-28 18:51:48.546209789 +0000 UTC m=+3059.622771787"
Jul 28 18:51:48.614291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:48.614254    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:48.619397 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:48.619369    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:49.530843 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:49.530790    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:49.535577 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:49.535543    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:51:50.554363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:50.554321    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:50.639771 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:50.639732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:50.695351 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:50.695313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:51.021482 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:51.021447    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:51.237234 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:51.237199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:51.336243 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:51.336151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:51:56.994283 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:51:56.994248    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:52:04.664652 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:04.664613    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": read tcp 10.132.0.2:48404->10.132.0.63:15021: read: connection reset by peer" start-of-body=
Jul 28 18:52:04.665101 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:04.664696    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": read tcp 10.132.0.2:48404->10.132.0.63:15021: read: connection reset by peer"
Jul 28 18:52:05.622193 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:05.622158    2583 generic.go:356] "Generic (PLEG): container finished" podID="3798bf77-2864-411e-8037-56e0d3d244b1" containerID="dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd" exitCode=0
Jul 28 18:52:05.622394 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:05.622213    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" event={"ID":"3798bf77-2864-411e-8037-56e0d3d244b1","Type":"ContainerDied","Data":"dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd"}
Jul 28 18:52:05.622645 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:05.622630    2583 scope.go:117] "RemoveContainer" containerID="dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd"
Jul 28 18:52:05.623673 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:05.623645    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:52:05.623738 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:05.623716    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:52:05.623774 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:05.623749    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:52:06.629483 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:06.629450    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" event={"ID":"3798bf77-2864-411e-8037-56e0d3d244b1","Type":"ContainerStarted","Data":"780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53"}
Jul 28 18:52:07.614315 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:07.614275    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:52:07.614644 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:07.614622    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 28 18:52:07.614739 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:07.614671    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 28 18:52:08.614515 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:08.614477    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 28 18:52:08.614951 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:08.614541    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 28 18:52:09.614740 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:09.614702    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 28 18:52:09.615176 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:09.614755    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 28 18:52:10.617982 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:10.617943    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:52:10.618363 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:10.618211    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:52:10.619000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:10.618981    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:52:15.488347 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:15.488314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:16.177330 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:16.177297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:16.478534 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:16.478493    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:16.698603 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:16.698570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:22.033376 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:22.033338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:22.130782 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:22.130745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:22.308553 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:22.307283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:22.447062 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:22.447010    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:22.752326 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:22.752294    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:22.858450 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:22.858421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:28.986233 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:28.986192    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:52:46.948862 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:46.948801    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:52:58.126295 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:52:58.126254    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:53:00.799641 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:00.799602    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:53:28.793971 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:28.793936    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:53:39.282077 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.282039    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:53:39.583101 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.583022    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"]
Jul 28 18:53:39.586684 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.586663    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.589525 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.589501    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-51b7e8-openshift-default-dockercfg-6v45r\""
Jul 28 18:53:39.597076 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.597051    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"]
Jul 28 18:53:39.605576 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.605555    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:53:39.660558 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.660508    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-certs\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.660735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.660565    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-data\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.660735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.660657    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/634dd7c1-b34e-44f9-8995-079e717e659b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.660735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.660697    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/634dd7c1-b34e-44f9-8995-079e717e659b-istio-podinfo\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.660885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.660775    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-envoy\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.660885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.660852    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-socket\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.660885 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.660875    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-istio-token\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.660981 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.660935    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9pmv\" (UniqueName: \"kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-kube-api-access-b9pmv\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.660981 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.660976    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-credential-socket\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762084 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762040    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-socket\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762084 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762087    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-istio-token\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762342 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762134    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9pmv\" (UniqueName: \"kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-kube-api-access-b9pmv\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762342 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762162    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-credential-socket\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762342 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762301    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-certs\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762342 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-data\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762541 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762398    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/634dd7c1-b34e-44f9-8995-079e717e659b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762541 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/634dd7c1-b34e-44f9-8995-079e717e659b-istio-podinfo\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762541 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762482    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-envoy\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762541 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762516    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-socket\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762767 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762543    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-credential-socket\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762767 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762703    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-certs\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.762767 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.762711    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-data\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.763128 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.763106    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/634dd7c1-b34e-44f9-8995-079e717e659b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.764768 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.764746    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-envoy\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.764923 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.764905    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/634dd7c1-b34e-44f9-8995-079e717e659b-istio-podinfo\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.770225 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.770199    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-istio-token\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.770341 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.770310    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9pmv\" (UniqueName: \"kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-kube-api-access-b9pmv\") pod \"e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") " pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:39.898215 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:39.898113    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:40.040042 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:40.040012    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"]
Jul 28 18:53:40.041844 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:53:40.041800    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634dd7c1_b34e_44f9_8995_079e717e659b.slice/crio-e8eb9b0c1e4679adc29c4164530b901d7e0f2ea23f923e5d035c4f2ad9e6d166 WatchSource:0}: Error finding container e8eb9b0c1e4679adc29c4164530b901d7e0f2ea23f923e5d035c4f2ad9e6d166: Status 404 returned error can't find the container with id e8eb9b0c1e4679adc29c4164530b901d7e0f2ea23f923e5d035c4f2ad9e6d166
Jul 28 18:53:40.044197 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:40.044161    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:53:40.044312 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:40.044251    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:53:40.044312 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:40.044294    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:53:40.115155 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:40.115126    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" event={"ID":"634dd7c1-b34e-44f9-8995-079e717e659b","Type":"ContainerStarted","Data":"54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3"}
Jul 28 18:53:40.115277 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:40.115175    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" event={"ID":"634dd7c1-b34e-44f9-8995-079e717e659b","Type":"ContainerStarted","Data":"e8eb9b0c1e4679adc29c4164530b901d7e0f2ea23f923e5d035c4f2ad9e6d166"}
Jul 28 18:53:40.136988 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:40.136931    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" podStartSLOduration=1.136911805 podStartE2EDuration="1.136911805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:53:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:53:40.134379081 +0000 UTC m=+3171.210941111" watchObservedRunningTime="2026-07-28 18:53:40.136911805 +0000 UTC m=+3171.213473802"
Jul 28 18:53:40.189423 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:40.189389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:53:40.898679 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:40.898641    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:41.904382 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:41.904350    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:42.063065 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:42.063016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:53:42.125874 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:42.125842    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:42.126807 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:42.126788    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:42.161324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:42.161234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:53:42.219237 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:42.219201    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:53:42.369502 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:42.369467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:53:42.728863 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:42.728831    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:53:42.828249 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:42.828202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:53:46.290000 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:46.289962    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:53:55.197547 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:55.197509    2583 generic.go:356] "Generic (PLEG): container finished" podID="634dd7c1-b34e-44f9-8995-079e717e659b" containerID="54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3" exitCode=0
Jul 28 18:53:55.198004 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:55.197558    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" event={"ID":"634dd7c1-b34e-44f9-8995-079e717e659b","Type":"ContainerDied","Data":"54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3"}
Jul 28 18:53:55.198004 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:55.197987    2583 scope.go:117] "RemoveContainer" containerID="54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3"
Jul 28 18:53:55.199186 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:55.199158    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:53:55.199265 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:55.199235    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:53:55.199308 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:55.199265    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:53:56.205042 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:56.205002    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" event={"ID":"634dd7c1-b34e-44f9-8995-079e717e659b","Type":"ContainerStarted","Data":"8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c"}
Jul 28 18:53:56.898920 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:56.898879    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:56.900148 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:56.900119    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 28 18:53:56.900270 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:56.900184    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 28 18:53:57.125990 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:57.125955    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:53:57.899262 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:57.899217    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 28 18:53:57.899656 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:57.899290    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 28 18:53:58.898808 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:58.898767    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 28 18:53:58.899016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:58.898868    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 28 18:53:59.898858 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:59.898801    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 28 18:53:59.899268 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:53:59.898908    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 28 18:54:00.903178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:00.903145    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:54:00.904244 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:00.904214    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:54:07.128132 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:07.128098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:12.829926 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:12.829894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:12.891783 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:12.891750    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:12.981735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:12.981699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:13.038160 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:13.038107    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:13.347674 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:13.347625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:17.396182 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:17.396139    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:54:19.224876 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:19.224842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:19.325613 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:19.325577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:19.480507 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:19.480423    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"]
Jul 28 18:54:19.480669 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:19.480648    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy" containerID="cri-o://8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c" gracePeriod=30
Jul 28 18:54:19.492968 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:19.492936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:19.723689 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:19.723648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:24.737142 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.737117    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:54:24.841354 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841267    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-certs\") pod \"634dd7c1-b34e-44f9-8995-079e717e659b\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") "
Jul 28 18:54:24.841354 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841324    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/634dd7c1-b34e-44f9-8995-079e717e659b-istiod-ca-cert\") pod \"634dd7c1-b34e-44f9-8995-079e717e659b\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") "
Jul 28 18:54:24.841354 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841354    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-credential-socket\") pod \"634dd7c1-b34e-44f9-8995-079e717e659b\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") "
Jul 28 18:54:24.841625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841391    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-data\") pod \"634dd7c1-b34e-44f9-8995-079e717e659b\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") "
Jul 28 18:54:24.841625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841433    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/634dd7c1-b34e-44f9-8995-079e717e659b-istio-podinfo\") pod \"634dd7c1-b34e-44f9-8995-079e717e659b\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") "
Jul 28 18:54:24.841625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841467    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-socket\") pod \"634dd7c1-b34e-44f9-8995-079e717e659b\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") "
Jul 28 18:54:24.841625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841517    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-envoy\") pod \"634dd7c1-b34e-44f9-8995-079e717e659b\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") "
Jul 28 18:54:24.841625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841608    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9pmv\" (UniqueName: \"kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-kube-api-access-b9pmv\") pod \"634dd7c1-b34e-44f9-8995-079e717e659b\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") "
Jul 28 18:54:24.841625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841608    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "634dd7c1-b34e-44f9-8995-079e717e659b" (UID: "634dd7c1-b34e-44f9-8995-079e717e659b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:24.842021 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841635    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-istio-token\") pod \"634dd7c1-b34e-44f9-8995-079e717e659b\" (UID: \"634dd7c1-b34e-44f9-8995-079e717e659b\") "
Jul 28 18:54:24.842021 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841801    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "634dd7c1-b34e-44f9-8995-079e717e659b" (UID: "634dd7c1-b34e-44f9-8995-079e717e659b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:24.842021 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841810    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "634dd7c1-b34e-44f9-8995-079e717e659b" (UID: "634dd7c1-b34e-44f9-8995-079e717e659b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:24.842021 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.841878    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/634dd7c1-b34e-44f9-8995-079e717e659b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "634dd7c1-b34e-44f9-8995-079e717e659b" (UID: "634dd7c1-b34e-44f9-8995-079e717e659b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:54:24.842248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.842138    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/634dd7c1-b34e-44f9-8995-079e717e659b-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:54:24.842248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.842157    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:54:24.842248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.842171    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:54:24.842248 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.842185    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:54:24.842539 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.842523    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-data" (OuterVolumeSpecName: "istio-data") pod "634dd7c1-b34e-44f9-8995-079e717e659b" (UID: "634dd7c1-b34e-44f9-8995-079e717e659b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:24.844266 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.844242    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "634dd7c1-b34e-44f9-8995-079e717e659b" (UID: "634dd7c1-b34e-44f9-8995-079e717e659b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:24.844266 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.844249    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/634dd7c1-b34e-44f9-8995-079e717e659b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "634dd7c1-b34e-44f9-8995-079e717e659b" (UID: "634dd7c1-b34e-44f9-8995-079e717e659b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:54:24.844458 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.844384    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-istio-token" (OuterVolumeSpecName: "istio-token") pod "634dd7c1-b34e-44f9-8995-079e717e659b" (UID: "634dd7c1-b34e-44f9-8995-079e717e659b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:54:24.844458 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.844411    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-kube-api-access-b9pmv" (OuterVolumeSpecName: "kube-api-access-b9pmv") pod "634dd7c1-b34e-44f9-8995-079e717e659b" (UID: "634dd7c1-b34e-44f9-8995-079e717e659b"). InnerVolumeSpecName "kube-api-access-b9pmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:54:24.943699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.943665    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:54:24.943699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.943692    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/634dd7c1-b34e-44f9-8995-079e717e659b-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:54:24.943699 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.943701    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/634dd7c1-b34e-44f9-8995-079e717e659b-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:54:24.943940 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.943711    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9pmv\" (UniqueName: \"kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-kube-api-access-b9pmv\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:54:24.943940 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:24.943720    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/634dd7c1-b34e-44f9-8995-079e717e659b-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:54:25.356610 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.356574    2583 generic.go:356] "Generic (PLEG): container finished" podID="634dd7c1-b34e-44f9-8995-079e717e659b" containerID="8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c" exitCode=0
Jul 28 18:54:25.356797 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.356622    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"
Jul 28 18:54:25.356797 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.356620    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" event={"ID":"634dd7c1-b34e-44f9-8995-079e717e659b","Type":"ContainerDied","Data":"8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c"}
Jul 28 18:54:25.356797 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.356671    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n" event={"ID":"634dd7c1-b34e-44f9-8995-079e717e659b","Type":"ContainerDied","Data":"e8eb9b0c1e4679adc29c4164530b901d7e0f2ea23f923e5d035c4f2ad9e6d166"}
Jul 28 18:54:25.356797 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.356692    2583 scope.go:117] "RemoveContainer" containerID="8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c"
Jul 28 18:54:25.366357 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.366339    2583 scope.go:117] "RemoveContainer" containerID="54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3"
Jul 28 18:54:25.374442 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.374423    2583 scope.go:117] "RemoveContainer" containerID="8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c"
Jul 28 18:54:25.374696 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:54:25.374674    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c\": container with ID starting with 8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c not found: ID does not exist" containerID="8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c"
Jul 28 18:54:25.374790 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.374702    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c"} err="failed to get container status \"8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c\": rpc error: code = NotFound desc = could not find container \"8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c\": container with ID starting with 8a46116c42b915cdbcbfe0674c82731d4cc946a9fedfec4737c17e3387f2416c not found: ID does not exist"
Jul 28 18:54:25.374790 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.374722    2583 scope.go:117] "RemoveContainer" containerID="54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3"
Jul 28 18:54:25.374986 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:54:25.374968    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3\": container with ID starting with 54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3 not found: ID does not exist" containerID="54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3"
Jul 28 18:54:25.375032 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.374992    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3"} err="failed to get container status \"54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3\": rpc error: code = NotFound desc = could not find container \"54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3\": container with ID starting with 54ce8df3323cc36e3913c3451c21b8f6aeb8212f178b17da16557a74e5740fa3 not found: ID does not exist"
Jul 28 18:54:25.381352 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.381327    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"]
Jul 28 18:54:25.387197 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.387166    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-51b7e8-openshift-default-698c954856-87d4n"]
Jul 28 18:54:25.405381 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:25.405354    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" path="/var/lib/kubelet/pods/634dd7c1-b34e-44f9-8995-079e717e659b/volumes"
Jul 28 18:54:38.497572 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:38.497539    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:54:38.659341 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:38.659291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:38.726405 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:38.726375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:38.789205 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:38.789102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:38.884261 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:38.884225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:39.226611 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:39.226581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:41.256500 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:41.256401    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:44.307514 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:44.307475    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:54:48.291465 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:54:48.291423    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:55:21.081602 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:21.081570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:55:21.414267 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:21.414240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:55:27.060936 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:27.060895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:55:49.612789 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:49.612757    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:55:49.617580 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:49.617558    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:55:50.430905 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:50.430871    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"]
Jul 28 18:55:50.431147 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:50.431104    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy" containerID="cri-o://780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53" gracePeriod=30
Jul 28 18:55:50.906629 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:50.906593    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:55:52.117396 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:52.117363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:55:55.693333 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.693274    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:55:55.760469 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760428    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-socket\") pod \"3798bf77-2864-411e-8037-56e0d3d244b1\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") "
Jul 28 18:55:55.760469 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760475    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-certs\") pod \"3798bf77-2864-411e-8037-56e0d3d244b1\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") "
Jul 28 18:55:55.760718 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760509    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-data\") pod \"3798bf77-2864-411e-8037-56e0d3d244b1\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") "
Jul 28 18:55:55.760718 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760572    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-credential-socket\") pod \"3798bf77-2864-411e-8037-56e0d3d244b1\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") "
Jul 28 18:55:55.760718 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760595    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3798bf77-2864-411e-8037-56e0d3d244b1-istiod-ca-cert\") pod \"3798bf77-2864-411e-8037-56e0d3d244b1\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") "
Jul 28 18:55:55.760718 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760634    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3798bf77-2864-411e-8037-56e0d3d244b1-istio-podinfo\") pod \"3798bf77-2864-411e-8037-56e0d3d244b1\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") "
Jul 28 18:55:55.760718 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760700    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2hsr\" (UniqueName: \"kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-kube-api-access-x2hsr\") pod \"3798bf77-2864-411e-8037-56e0d3d244b1\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") "
Jul 28 18:55:55.761030 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760731    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-envoy\") pod \"3798bf77-2864-411e-8037-56e0d3d244b1\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") "
Jul 28 18:55:55.761030 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760769    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-istio-token\") pod \"3798bf77-2864-411e-8037-56e0d3d244b1\" (UID: \"3798bf77-2864-411e-8037-56e0d3d244b1\") "
Jul 28 18:55:55.761030 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760726    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3798bf77-2864-411e-8037-56e0d3d244b1" (UID: "3798bf77-2864-411e-8037-56e0d3d244b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:55:55.761214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.761062    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3798bf77-2864-411e-8037-56e0d3d244b1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3798bf77-2864-411e-8037-56e0d3d244b1" (UID: "3798bf77-2864-411e-8037-56e0d3d244b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:55.761214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760887    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3798bf77-2864-411e-8037-56e0d3d244b1" (UID: "3798bf77-2864-411e-8037-56e0d3d244b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:55:55.761214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.760927    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3798bf77-2864-411e-8037-56e0d3d244b1" (UID: "3798bf77-2864-411e-8037-56e0d3d244b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:55:55.761214 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.761203    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:55:55.761430 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.761219    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3798bf77-2864-411e-8037-56e0d3d244b1-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:55:55.761430 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.761230    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:55:55.761430 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.761239    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:55:55.761612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.761584    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-data" (OuterVolumeSpecName: "istio-data") pod "3798bf77-2864-411e-8037-56e0d3d244b1" (UID: "3798bf77-2864-411e-8037-56e0d3d244b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:55:55.763013 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.762982    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3798bf77-2864-411e-8037-56e0d3d244b1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3798bf77-2864-411e-8037-56e0d3d244b1" (UID: "3798bf77-2864-411e-8037-56e0d3d244b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:55:55.763152 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.763131    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-istio-token" (OuterVolumeSpecName: "istio-token") pod "3798bf77-2864-411e-8037-56e0d3d244b1" (UID: "3798bf77-2864-411e-8037-56e0d3d244b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:55.763241 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.763225    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3798bf77-2864-411e-8037-56e0d3d244b1" (UID: "3798bf77-2864-411e-8037-56e0d3d244b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:55:55.763372 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.763359    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-kube-api-access-x2hsr" (OuterVolumeSpecName: "kube-api-access-x2hsr") pod "3798bf77-2864-411e-8037-56e0d3d244b1" (UID: "3798bf77-2864-411e-8037-56e0d3d244b1"). InnerVolumeSpecName "kube-api-access-x2hsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:55.829456 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.829419    2583 generic.go:356] "Generic (PLEG): container finished" podID="3798bf77-2864-411e-8037-56e0d3d244b1" containerID="780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53" exitCode=0
Jul 28 18:55:55.829602 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.829472    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"
Jul 28 18:55:55.829602 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.829501    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" event={"ID":"3798bf77-2864-411e-8037-56e0d3d244b1","Type":"ContainerDied","Data":"780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53"}
Jul 28 18:55:55.829602 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.829558    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" event={"ID":"3798bf77-2864-411e-8037-56e0d3d244b1","Type":"ContainerDied","Data":"a2fc1bb2d6a2b2eccb1bae21c46c6461bd0d632dc9548b14cc493298f801bb38"}
Jul 28 18:55:55.829602 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.829581    2583 scope.go:117] "RemoveContainer" containerID="780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53"
Jul 28 18:55:55.839720 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.839705    2583 scope.go:117] "RemoveContainer" containerID="dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd"
Jul 28 18:55:55.847751 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.847735    2583 scope.go:117] "RemoveContainer" containerID="780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53"
Jul 28 18:55:55.848017 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:55:55.848001    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53\": container with ID starting with 780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53 not found: ID does not exist" containerID="780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53"
Jul 28 18:55:55.848066 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.848024    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53"} err="failed to get container status \"780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53\": rpc error: code = NotFound desc = could not find container \"780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53\": container with ID starting with 780d3d8aaacd2c3c94111bcfb6ac89c20880457e9aa743b490cd5f44b6712b53 not found: ID does not exist"
Jul 28 18:55:55.848066 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.848041    2583 scope.go:117] "RemoveContainer" containerID="dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd"
Jul 28 18:55:55.848254 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:55:55.848236    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd\": container with ID starting with dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd not found: ID does not exist" containerID="dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd"
Jul 28 18:55:55.848324 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.848263    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd"} err="failed to get container status \"dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd\": rpc error: code = NotFound desc = could not find container \"dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd\": container with ID starting with dd67a740e7ce0f4d392d0f0168708f6f222996c80e85183385751d402c581bfd not found: ID does not exist"
Jul 28 18:55:55.851787 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.851763    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"]
Jul 28 18:55:55.854895 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.854877    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg"]
Jul 28 18:55:55.862270 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.862253    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:55:55.862335 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.862273    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3798bf77-2864-411e-8037-56e0d3d244b1-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:55:55.862335 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.862283    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2hsr\" (UniqueName: \"kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-kube-api-access-x2hsr\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:55:55.862335 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.862292    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3798bf77-2864-411e-8037-56e0d3d244b1-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:55:55.862335 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:55.862301    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3798bf77-2864-411e-8037-56e0d3d244b1-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:55:56.619806 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:56.619766    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 18:55:56.620010 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:56.619845    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-c1237e-openshift-default-5cd5754668-6x4dg" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 18:55:57.405175 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:55:57.405139    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" path="/var/lib/kubelet/pods/3798bf77-2864-411e-8037-56e0d3d244b1/volumes"
Jul 28 18:56:01.222036 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:01.222002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:01.281511 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:01.281478    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:01.343526 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:01.343481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:01.400918 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:01.400878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:01.764056 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:01.764016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:04.595025 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:04.594996    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:56:07.573222 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:07.573187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:07.673538 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:07.673502    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:07.880487 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:07.880374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:08.159156 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:08.159068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:11.887410 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:11.887330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:18.743935 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:18.743899    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:18.806358 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:18.806317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:18.865082 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:18.865034    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:18.925422 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:18.925374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:19.244674 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:19.244642    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:21.595101 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:21.595060    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:56:25.244965 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:25.244930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:25.345177 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:25.345131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:25.493206 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:25.493173    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"]
Jul 28 18:56:25.493468 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:25.493420    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" podUID="319e4143-3d67-48e6-abad-7be053f859e9" containerName="istio-proxy" containerID="cri-o://7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441" gracePeriod=30
Jul 28 18:56:25.511687 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:25.511602    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:25.745769 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:25.745721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:30.746749 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.746716    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:56:30.797807 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.797721    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-certs\") pod \"319e4143-3d67-48e6-abad-7be053f859e9\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") "
Jul 28 18:56:30.797807 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.797770    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-credential-socket\") pod \"319e4143-3d67-48e6-abad-7be053f859e9\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") "
Jul 28 18:56:30.798050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.797809    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcfmz\" (UniqueName: \"kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-kube-api-access-pcfmz\") pod \"319e4143-3d67-48e6-abad-7be053f859e9\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") "
Jul 28 18:56:30.798050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.797857    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-envoy\") pod \"319e4143-3d67-48e6-abad-7be053f859e9\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") "
Jul 28 18:56:30.798050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.797886    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-socket\") pod \"319e4143-3d67-48e6-abad-7be053f859e9\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") "
Jul 28 18:56:30.798050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.797921    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-istio-token\") pod \"319e4143-3d67-48e6-abad-7be053f859e9\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") "
Jul 28 18:56:30.798050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.797962    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/319e4143-3d67-48e6-abad-7be053f859e9-istiod-ca-cert\") pod \"319e4143-3d67-48e6-abad-7be053f859e9\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") "
Jul 28 18:56:30.798050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.797994    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-data\") pod \"319e4143-3d67-48e6-abad-7be053f859e9\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") "
Jul 28 18:56:30.798050 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.798024    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/319e4143-3d67-48e6-abad-7be053f859e9-istio-podinfo\") pod \"319e4143-3d67-48e6-abad-7be053f859e9\" (UID: \"319e4143-3d67-48e6-abad-7be053f859e9\") "
Jul 28 18:56:30.798431 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.798053    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "319e4143-3d67-48e6-abad-7be053f859e9" (UID: "319e4143-3d67-48e6-abad-7be053f859e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:30.798431 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.798162    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "319e4143-3d67-48e6-abad-7be053f859e9" (UID: "319e4143-3d67-48e6-abad-7be053f859e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:30.798536 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.798440    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/319e4143-3d67-48e6-abad-7be053f859e9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "319e4143-3d67-48e6-abad-7be053f859e9" (UID: "319e4143-3d67-48e6-abad-7be053f859e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:56:30.798536 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.798450    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:30.798536 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.798471    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:30.798698 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.798541    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "319e4143-3d67-48e6-abad-7be053f859e9" (UID: "319e4143-3d67-48e6-abad-7be053f859e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:30.799209 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.799181    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-data" (OuterVolumeSpecName: "istio-data") pod "319e4143-3d67-48e6-abad-7be053f859e9" (UID: "319e4143-3d67-48e6-abad-7be053f859e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:30.800170 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.800147    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-kube-api-access-pcfmz" (OuterVolumeSpecName: "kube-api-access-pcfmz") pod "319e4143-3d67-48e6-abad-7be053f859e9" (UID: "319e4143-3d67-48e6-abad-7be053f859e9"). InnerVolumeSpecName "kube-api-access-pcfmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:56:30.800343 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.800317    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/319e4143-3d67-48e6-abad-7be053f859e9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "319e4143-3d67-48e6-abad-7be053f859e9" (UID: "319e4143-3d67-48e6-abad-7be053f859e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:56:30.800483 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.800457    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "319e4143-3d67-48e6-abad-7be053f859e9" (UID: "319e4143-3d67-48e6-abad-7be053f859e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:30.800673 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.800653    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-istio-token" (OuterVolumeSpecName: "istio-token") pod "319e4143-3d67-48e6-abad-7be053f859e9" (UID: "319e4143-3d67-48e6-abad-7be053f859e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:56:30.899278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.899242    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:30.899278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.899274    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/319e4143-3d67-48e6-abad-7be053f859e9-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:30.899278 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.899284    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:30.899519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.899292    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/319e4143-3d67-48e6-abad-7be053f859e9-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:30.899519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.899301    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcfmz\" (UniqueName: \"kubernetes.io/projected/319e4143-3d67-48e6-abad-7be053f859e9-kube-api-access-pcfmz\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:30.899519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.899310    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:30.899519 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:30.899318    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/319e4143-3d67-48e6-abad-7be053f859e9-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:31.022266 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.022227    2583 generic.go:356] "Generic (PLEG): container finished" podID="319e4143-3d67-48e6-abad-7be053f859e9" containerID="7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441" exitCode=0
Jul 28 18:56:31.022453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.022286    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"
Jul 28 18:56:31.022453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.022309    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" event={"ID":"319e4143-3d67-48e6-abad-7be053f859e9","Type":"ContainerDied","Data":"7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441"}
Jul 28 18:56:31.022453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.022365    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj" event={"ID":"319e4143-3d67-48e6-abad-7be053f859e9","Type":"ContainerDied","Data":"122aa7d03d635c3438714ffd91f4f5ba37ddab148db0f88138932fc68c204e31"}
Jul 28 18:56:31.022453 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.022388    2583 scope.go:117] "RemoveContainer" containerID="7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441"
Jul 28 18:56:31.033005 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.032475    2583 scope.go:117] "RemoveContainer" containerID="acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25"
Jul 28 18:56:31.041063 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.041043    2583 scope.go:117] "RemoveContainer" containerID="7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441"
Jul 28 18:56:31.041296 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:56:31.041277    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441\": container with ID starting with 7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441 not found: ID does not exist" containerID="7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441"
Jul 28 18:56:31.041343 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.041306    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441"} err="failed to get container status \"7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441\": rpc error: code = NotFound desc = could not find container \"7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441\": container with ID starting with 7ba8fb2d87f5de9665cabb61f72c7603141bdcb0a7a5e98f990f9c8f2eea9441 not found: ID does not exist"
Jul 28 18:56:31.041343 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.041322    2583 scope.go:117] "RemoveContainer" containerID="acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25"
Jul 28 18:56:31.041526 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:56:31.041509    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25\": container with ID starting with acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25 not found: ID does not exist" containerID="acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25"
Jul 28 18:56:31.041570 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.041532    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25"} err="failed to get container status \"acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25\": rpc error: code = NotFound desc = could not find container \"acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25\": container with ID starting with acc963f1fe25eb48be541b3ebe615741085cf1ab7a3f2485d3a3aefa35509b25 not found: ID does not exist"
Jul 28 18:56:31.048568 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.048516    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"]
Jul 28 18:56:31.054095 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.054074    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-37e11d-openshift-default-6664b989b-h4ckj"]
Jul 28 18:56:31.405910 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:31.405831    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="319e4143-3d67-48e6-abad-7be053f859e9" path="/var/lib/kubelet/pods/319e4143-3d67-48e6-abad-7be053f859e9/volumes"
Jul 28 18:56:36.488281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:36.488207    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:36.545325 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:36.545278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:36.634016 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:36.633978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:36.692093 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:36.692059    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:37.015219 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:37.015183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:39.489145 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:39.489109    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6zqbp"]
Jul 28 18:56:43.002772 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:43.002728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:43.060139 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:43.060106    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:43.214868 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:43.214829    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"]
Jul 28 18:56:43.215153 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:43.215125    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy" containerID="cri-o://bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150" gracePeriod=30
Jul 28 18:56:43.231839 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:43.231798    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:43.461995 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:43.461959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:56:48.469098 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.469063    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:56:48.475447 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475422    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-token\") pod \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") "
Jul 28 18:56:48.475555 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475471    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-credential-socket\") pod \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") "
Jul 28 18:56:48.475555 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475492    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istiod-ca-cert\") pod \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") "
Jul 28 18:56:48.475555 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475517    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-socket\") pod \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") "
Jul 28 18:56:48.475732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475561    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-podinfo\") pod \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") "
Jul 28 18:56:48.475732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475581    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96654\" (UniqueName: \"kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-kube-api-access-96654\") pod \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") "
Jul 28 18:56:48.475732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475618    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-certs\") pod \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") "
Jul 28 18:56:48.475732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475652    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-envoy\") pod \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") "
Jul 28 18:56:48.475732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475677    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-data\") pod \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\" (UID: \"2c74e277-d844-43ed-8ab9-fcdd2b57376f\") "
Jul 28 18:56:48.475732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475696    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c74e277-d844-43ed-8ab9-fcdd2b57376f" (UID: "2c74e277-d844-43ed-8ab9-fcdd2b57376f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:48.476178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475959    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c74e277-d844-43ed-8ab9-fcdd2b57376f" (UID: "2c74e277-d844-43ed-8ab9-fcdd2b57376f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:48.476178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.475966    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c74e277-d844-43ed-8ab9-fcdd2b57376f" (UID: "2c74e277-d844-43ed-8ab9-fcdd2b57376f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:56:48.476178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.476056    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-certs\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:48.476178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.476073    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-credential-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:48.476178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.476087    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istiod-ca-cert\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:48.476178 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.476151    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c74e277-d844-43ed-8ab9-fcdd2b57376f" (UID: "2c74e277-d844-43ed-8ab9-fcdd2b57376f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:48.476727 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.476706    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c74e277-d844-43ed-8ab9-fcdd2b57376f" (UID: "2c74e277-d844-43ed-8ab9-fcdd2b57376f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:48.478365 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.478331    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-kube-api-access-96654" (OuterVolumeSpecName: "kube-api-access-96654") pod "2c74e277-d844-43ed-8ab9-fcdd2b57376f" (UID: "2c74e277-d844-43ed-8ab9-fcdd2b57376f"). InnerVolumeSpecName "kube-api-access-96654". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:56:48.478365 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.478364    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c74e277-d844-43ed-8ab9-fcdd2b57376f" (UID: "2c74e277-d844-43ed-8ab9-fcdd2b57376f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:56:48.478667 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.478647    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c74e277-d844-43ed-8ab9-fcdd2b57376f" (UID: "2c74e277-d844-43ed-8ab9-fcdd2b57376f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:56:48.478750 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.478649    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c74e277-d844-43ed-8ab9-fcdd2b57376f" (UID: "2c74e277-d844-43ed-8ab9-fcdd2b57376f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:56:48.576512 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.576482    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-token\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:48.576512 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.576508    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-workload-socket\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:48.576512 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.576518    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-podinfo\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:48.576732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.576527    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96654\" (UniqueName: \"kubernetes.io/projected/2c74e277-d844-43ed-8ab9-fcdd2b57376f-kube-api-access-96654\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:48.576732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.576537    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-envoy\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:48.576732 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:48.576545    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c74e277-d844-43ed-8ab9-fcdd2b57376f-istio-data\") on node \"ip-10-0-141-192.ec2.internal\" DevicePath \"\""
Jul 28 18:56:49.114303 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.114266    2583 generic.go:356] "Generic (PLEG): container finished" podID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerID="bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150" exitCode=0
Jul 28 18:56:49.114458 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.114319    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"
Jul 28 18:56:49.114458 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.114324    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" event={"ID":"2c74e277-d844-43ed-8ab9-fcdd2b57376f","Type":"ContainerDied","Data":"bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150"}
Jul 28 18:56:49.114458 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.114376    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn" event={"ID":"2c74e277-d844-43ed-8ab9-fcdd2b57376f","Type":"ContainerDied","Data":"897e5b2dcab95522115f86f9029d018f5357d467a682f0564d10214f4bed800c"}
Jul 28 18:56:49.114458 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.114408    2583 scope.go:117] "RemoveContainer" containerID="bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150"
Jul 28 18:56:49.126507 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.126479    2583 scope.go:117] "RemoveContainer" containerID="2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54"
Jul 28 18:56:49.136006 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.135985    2583 scope.go:117] "RemoveContainer" containerID="bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150"
Jul 28 18:56:49.136322 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:56:49.136295    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150\": container with ID starting with bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150 not found: ID does not exist" containerID="bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150"
Jul 28 18:56:49.136392 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.136335    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150"} err="failed to get container status \"bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150\": rpc error: code = NotFound desc = could not find container \"bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150\": container with ID starting with bc444b44a4d9312ba59ec15fdcdea99dd7363137426e6d7332e397e026ba8150 not found: ID does not exist"
Jul 28 18:56:49.136392 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.136360    2583 scope.go:117] "RemoveContainer" containerID="2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54"
Jul 28 18:56:49.136658 ip-10-0-141-192 kubenswrapper[2583]: E0728 18:56:49.136635    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54\": container with ID starting with 2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54 not found: ID does not exist" containerID="2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54"
Jul 28 18:56:49.136708 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.136669    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54"} err="failed to get container status \"2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54\": rpc error: code = NotFound desc = could not find container \"2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54\": container with ID starting with 2db196d6b7865e4bb74512654f6ab9486bd1e07fac21cf248f272f7c5cadbb54 not found: ID does not exist"
Jul 28 18:56:49.137590 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.137571    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"]
Jul 28 18:56:49.141442 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.141421    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f36101-openshift-default-5fb99f6f5b-lt6jn"]
Jul 28 18:56:49.407434 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.407357    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" path="/var/lib/kubelet/pods/2c74e277-d844-43ed-8ab9-fcdd2b57376f/volumes"
Jul 28 18:56:49.963670 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.963643    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7795fd7c74-rxz9c_eb8a4236-d236-4a19-aced-a4778d0512e1/copy-manifests/0.log"
Jul 28 18:56:49.980497 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:49.980468    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7795fd7c74-rxz9c_eb8a4236-d236-4a19-aced-a4778d0512e1/manager/0.log"
Jul 28 18:56:52.262960 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:52.262932    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nwqx2_3e52f160-84d4-419d-a243-a31a003fdbd2/manager/0.log"
Jul 28 18:56:52.377014 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:52.376985    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8k25x_c77998b7-160c-40bd-8b54-61ee03492679/manager/0.log"
Jul 28 18:56:52.616138 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:52.616060    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-878fw_6c4b138e-70e7-4fe6-bb25-c5e88cf74ac8/registry-server/0.log"
Jul 28 18:56:52.736746 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:52.736709    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wz2tr_a409a38d-bfb3-4080-9141-8ca70955c03e/manager/0.log"
Jul 28 18:56:52.847191 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:52.847158    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6zqbp_b0da8a38-9108-4c77-a3fc-dfc0c41c52c4/limitador/0.log"
Jul 28 18:56:53.297790 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:53.297757    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cqskvj_039a55ec-d600-4f19-b3ad-76642f0b42b2/istio-proxy/0.log"
Jul 28 18:56:53.757522 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:53.757485    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jlmbq_ea3a4c2a-2d05-4ce3-8a73-5fbcae566368/istio-proxy/1.log"
Jul 28 18:56:54.119581 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:54.119498    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cb56ffbff-lv2ps_e6b6c4c5-a304-4406-87eb-e99a3e1f3453/router/0.log"
Jul 28 18:56:54.800735 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:56:54.800706    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnqdr_f31a69aa-9651-4768-88c5-f9e394b3ddb6/main/0.log"
Jul 28 18:57:00.461956 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:00.461918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:57:00.561723 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:00.561661    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log" limit=1024
Jul 28 18:57:03.171263 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:03.171233    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tc422_1d6960ee-cc72-488e-bae6-23608c90ab15/global-pull-secret-syncer/0.log"
Jul 28 18:57:03.227777 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:03.227745    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-f6hcm_59fb8d62-c741-4255-88c7-bc5d23e182a9/konnectivity-agent/0.log"
Jul 28 18:57:03.381804 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:03.381771    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-192.ec2.internal_54337df22542d0d892eb9498fe899d87/haproxy/0.log"
Jul 28 18:57:08.550291 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:08.550258    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nwqx2_3e52f160-84d4-419d-a243-a31a003fdbd2/manager/0.log"
Jul 28 18:57:08.591938 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:08.591915    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8k25x_c77998b7-160c-40bd-8b54-61ee03492679/manager/0.log"
Jul 28 18:57:08.669015 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:08.668980    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-878fw_6c4b138e-70e7-4fe6-bb25-c5e88cf74ac8/registry-server/0.log"
Jul 28 18:57:08.885263 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:08.885179    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wz2tr_a409a38d-bfb3-4080-9141-8ca70955c03e/manager/0.log"
Jul 28 18:57:08.918466 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:08.918440    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6zqbp_b0da8a38-9108-4c77-a3fc-dfc0c41c52c4/limitador/0.log"
Jul 28 18:57:10.704348 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:10.704315    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b25b020d-dea5-4c28-b08a-a1c7db50a5ea/alertmanager/0.log"
Jul 28 18:57:10.727303 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:10.727280    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b25b020d-dea5-4c28-b08a-a1c7db50a5ea/config-reloader/0.log"
Jul 28 18:57:10.746126 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:10.746105    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b25b020d-dea5-4c28-b08a-a1c7db50a5ea/kube-rbac-proxy-web/0.log"
Jul 28 18:57:10.770683 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:10.770662    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b25b020d-dea5-4c28-b08a-a1c7db50a5ea/kube-rbac-proxy/0.log"
Jul 28 18:57:10.789883 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:10.789860    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b25b020d-dea5-4c28-b08a-a1c7db50a5ea/kube-rbac-proxy-metric/0.log"
Jul 28 18:57:10.811253 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:10.811191    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b25b020d-dea5-4c28-b08a-a1c7db50a5ea/prom-label-proxy/0.log"
Jul 28 18:57:10.831612 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:10.831592    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b25b020d-dea5-4c28-b08a-a1c7db50a5ea/init-config-reloader/0.log"
Jul 28 18:57:10.896376 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:10.896351    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-zhfld_ce9fc0bb-49b9-4598-9624-c944ce193418/cluster-monitoring-operator/0.log"
Jul 28 18:57:10.988532 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:10.988497    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-567db57b8d-d65mx_76fa890b-57e3-4aa7-b6b8-73e325fac8f2/metrics-server/0.log"
Jul 28 18:57:11.018597 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.018569    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-bvfmg_0a6460a8-9d08-4c3a-98aa-726704e52594/monitoring-plugin/0.log"
Jul 28 18:57:11.060281 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.060244    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5q6wc_ca29f8de-1c36-49da-960e-cf14ae3c261e/node-exporter/0.log"
Jul 28 18:57:11.074155 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.074090    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5q6wc_ca29f8de-1c36-49da-960e-cf14ae3c261e/kube-rbac-proxy/0.log"
Jul 28 18:57:11.092418 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.092389    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5q6wc_ca29f8de-1c36-49da-960e-cf14ae3c261e/init-textfile/0.log"
Jul 28 18:57:11.338184 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.338098    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a36fd72a-e1bb-4048-8be3-ddb7244c0cbf/prometheus/0.log"
Jul 28 18:57:11.356764 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.356735    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a36fd72a-e1bb-4048-8be3-ddb7244c0cbf/config-reloader/0.log"
Jul 28 18:57:11.377491 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.377472    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a36fd72a-e1bb-4048-8be3-ddb7244c0cbf/thanos-sidecar/0.log"
Jul 28 18:57:11.400434 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.400412    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a36fd72a-e1bb-4048-8be3-ddb7244c0cbf/kube-rbac-proxy-web/0.log"
Jul 28 18:57:11.418469 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.418445    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a36fd72a-e1bb-4048-8be3-ddb7244c0cbf/kube-rbac-proxy/0.log"
Jul 28 18:57:11.436209 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.436188    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a36fd72a-e1bb-4048-8be3-ddb7244c0cbf/kube-rbac-proxy-thanos/0.log"
Jul 28 18:57:11.453962 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.453942    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a36fd72a-e1bb-4048-8be3-ddb7244c0cbf/init-config-reloader/0.log"
Jul 28 18:57:11.480403 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.480380    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-dhpjg_f9a3bc76-128c-4c0c-b4ab-1bf171726e0d/prometheus-operator/0.log"
Jul 28 18:57:11.505030 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.505007    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-dhpjg_f9a3bc76-128c-4c0c-b4ab-1bf171726e0d/kube-rbac-proxy/0.log"
Jul 28 18:57:11.525615 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:11.525594    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-n8ccv_b50cec69-24be-4ce0-b23d-89ced53eac8a/prometheus-operator-admission-webhook/0.log"
Jul 28 18:57:13.062650 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.062616    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"]
Jul 28 18:57:13.063094 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063039    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy"
Jul 28 18:57:13.063094 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063058    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy"
Jul 28 18:57:13.063094 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063074    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="319e4143-3d67-48e6-abad-7be053f859e9" containerName="istio-proxy"
Jul 28 18:57:13.063094 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063081    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="319e4143-3d67-48e6-abad-7be053f859e9" containerName="istio-proxy"
Jul 28 18:57:13.063094 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063093    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063101    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063116    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063124    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063141    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063149    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063159    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063167    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063183    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063188    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063257    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="319e4143-3d67-48e6-abad-7be053f859e9" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063265    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063271    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="319e4143-3d67-48e6-abad-7be053f859e9" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063276    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="634dd7c1-b34e-44f9-8995-079e717e659b" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063284    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063291    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c74e277-d844-43ed-8ab9-fcdd2b57376f" containerName="istio-proxy"
Jul 28 18:57:13.063290 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063297    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy"
Jul 28 18:57:13.063765 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.063305    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3798bf77-2864-411e-8037-56e0d3d244b1" containerName="istio-proxy"
Jul 28 18:57:13.066535 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.066519    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.069594 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.069570    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-22fmk\"/\"kube-root-ca.crt\""
Jul 28 18:57:13.069594 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.069582    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-22fmk\"/\"openshift-service-ca.crt\""
Jul 28 18:57:13.069764 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.069694    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-22fmk\"/\"default-dockercfg-fc5m6\""
Jul 28 18:57:13.077220 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.077197    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"]
Jul 28 18:57:13.197390 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.197351    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-sys\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.197556 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.197398    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvlhx\" (UniqueName: \"kubernetes.io/projected/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-kube-api-access-rvlhx\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.197556 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.197416    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-lib-modules\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.197556 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.197478    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-proc\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.197556 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.197551    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-podres\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.298376 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.298343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvlhx\" (UniqueName: \"kubernetes.io/projected/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-kube-api-access-rvlhx\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.298534 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.298388    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-lib-modules\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.298534 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.298411    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-proc\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.298534 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.298463    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-podres\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.298711 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.298527    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-proc\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.298711 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.298563    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-sys\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.298711 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.298578    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-lib-modules\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.298711 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.298639    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-sys\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.298711 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.298643    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-podres\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.306652 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.306628    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvlhx\" (UniqueName: \"kubernetes.io/projected/adfe09e0-3dc5-47ed-948e-4cccde8a8d0e-kube-api-access-rvlhx\") pod \"perf-node-gather-daemonset-gb7v6\" (UID: \"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e\") " pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.376463 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.376383    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:13.442228 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.442198    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/2.log"
Jul 28 18:57:13.453570 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.453538    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-t7pms_abbcaca7-e6ea-4ef4-b382-2d3c38ff18f6/console-operator/3.log"
Jul 28 18:57:13.513625 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:13.513600    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"]
Jul 28 18:57:13.515886 ip-10-0-141-192 kubenswrapper[2583]: W0728 18:57:13.515860    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podadfe09e0_3dc5_47ed_948e_4cccde8a8d0e.slice/crio-e83ee90d08613abecd63b1ed548c1a81601095e80f738266bc89784f91a2f419 WatchSource:0}: Error finding container e83ee90d08613abecd63b1ed548c1a81601095e80f738266bc89784f91a2f419: Status 404 returned error can't find the container with id e83ee90d08613abecd63b1ed548c1a81601095e80f738266bc89784f91a2f419
Jul 28 18:57:14.234874 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:14.234838    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6" event={"ID":"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e","Type":"ContainerStarted","Data":"4b5795e89bbaab54c7fe7897be227f129deeb5be1e9cb610ef2183fc0e9cf34c"}
Jul 28 18:57:14.235274 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:14.234898    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:14.235274 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:14.234919    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6" event={"ID":"adfe09e0-3dc5-47ed-948e-4cccde8a8d0e","Type":"ContainerStarted","Data":"e83ee90d08613abecd63b1ed548c1a81601095e80f738266bc89784f91a2f419"}
Jul 28 18:57:14.251620 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:14.251577    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6" podStartSLOduration=1.251562658 podStartE2EDuration="1.251562658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:57:14.250877333 +0000 UTC m=+3385.327439323" watchObservedRunningTime="2026-07-28 18:57:14.251562658 +0000 UTC m=+3385.328124654"
Jul 28 18:57:14.588518 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:14.588435    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-l7tlv_fa0ea14d-3bf0-4de4-ab96-2cc098eda35d/volume-data-source-validator/0.log"
Jul 28 18:57:15.564106 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:15.564078    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ftg8z_fd9c632b-0b01-4a1d-a636-5ce8ec009f9c/dns/0.log"
Jul 28 18:57:15.582026 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:15.582004    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ftg8z_fd9c632b-0b01-4a1d-a636-5ce8ec009f9c/kube-rbac-proxy/0.log"
Jul 28 18:57:15.711218 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:15.711189    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dfk6x_d8a76901-42f5-413e-8fa4-64c94fe7bcf7/dns-node-resolver/0.log"
Jul 28 18:57:16.282649 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:16.282621    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qz6v8_6887fd7f-e61b-49b4-8581-a50259ab2be2/node-ca/0.log"
Jul 28 18:57:17.215950 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:17.215914    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cqskvj_039a55ec-d600-4f19-b3ad-76642f0b42b2/istio-proxy/0.log"
Jul 28 18:57:17.423402 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:17.423371    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jlmbq_ea3a4c2a-2d05-4ce3-8a73-5fbcae566368/istio-proxy/0.log"
Jul 28 18:57:17.481930 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:17.477319    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jlmbq_ea3a4c2a-2d05-4ce3-8a73-5fbcae566368/istio-proxy/1.log"
Jul 28 18:57:17.567423 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:17.567387    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cb56ffbff-lv2ps_e6b6c4c5-a304-4406-87eb-e99a3e1f3453/router/0.log"
Jul 28 18:57:18.373759 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:18.373725    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wtkbg_b3fb8585-f057-4f84-9576-14df70802b01/serve-healthcheck-canary/0.log"
Jul 28 18:57:18.817105 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:18.817070    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-7p6xv_bb04a323-a317-43c8-b604-d817792bdf73/insights-operator/0.log"
Jul 28 18:57:18.819274 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:18.819251    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-7p6xv_bb04a323-a317-43c8-b604-d817792bdf73/insights-operator/1.log"
Jul 28 18:57:18.841317 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:18.841291    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5nxds_10f0792e-73a1-45ed-93fd-508e7885060d/kube-rbac-proxy/0.log"
Jul 28 18:57:18.860607 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:18.860582    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5nxds_10f0792e-73a1-45ed-93fd-508e7885060d/exporter/0.log"
Jul 28 18:57:18.881235 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:18.881204    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5nxds_10f0792e-73a1-45ed-93fd-508e7885060d/extractor/0.log"
Jul 28 18:57:20.249489 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:20.249463    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-22fmk/perf-node-gather-daemonset-gb7v6"
Jul 28 18:57:22.006289 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:22.006207    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7795fd7c74-rxz9c_eb8a4236-d236-4a19-aced-a4778d0512e1/manager/0.log"
Jul 28 18:57:22.022616 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:22.022596    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7795fd7c74-rxz9c_eb8a4236-d236-4a19-aced-a4778d0512e1/copy-manifests/0.log"
Jul 28 18:57:23.333428 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:23.333395    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cqskvj_039a55ec-d600-4f19-b3ad-76642f0b42b2/istio-proxy/0.log"
Jul 28 18:57:23.489125 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:23.489095    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84df6f5b9b-vb59l_a8ab0c02-5d57-4e79-863d-2000d0119e5a/manager/0.log"
Jul 28 18:57:23.515831 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:23.515785    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-d27x7_5fd2a3ed-6a11-4158-902f-e0ff128027c4/openshift-lws-operator/0.log"
Jul 28 18:57:23.516319 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:23.516295    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jlmbq_ea3a4c2a-2d05-4ce3-8a73-5fbcae566368/istio-proxy/0.log"
Jul 28 18:57:23.570125 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:23.570087    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jlmbq_ea3a4c2a-2d05-4ce3-8a73-5fbcae566368/istio-proxy/1.log"
Jul 28 18:57:23.624533 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:23.624505    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cb56ffbff-lv2ps_e6b6c4c5-a304-4406-87eb-e99a3e1f3453/router/0.log"
Jul 28 18:57:24.941992 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:24.941961    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cqskvj_039a55ec-d600-4f19-b3ad-76642f0b42b2/istio-proxy/0.log"
Jul 28 18:57:25.130279 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:25.130199    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jlmbq_ea3a4c2a-2d05-4ce3-8a73-5fbcae566368/istio-proxy/0.log"
Jul 28 18:57:25.181595 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:25.181563    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-jlmbq_ea3a4c2a-2d05-4ce3-8a73-5fbcae566368/istio-proxy/1.log"
Jul 28 18:57:25.244563 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:25.244520    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cb56ffbff-lv2ps_e6b6c4c5-a304-4406-87eb-e99a3e1f3453/router/0.log"
Jul 28 18:57:30.022062 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:30.022036    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cl2r6_65b87c8d-03ec-45d8-8d7e-76892c63898c/kube-multus-additional-cni-plugins/0.log"
Jul 28 18:57:30.043344 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:30.043320    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cl2r6_65b87c8d-03ec-45d8-8d7e-76892c63898c/egress-router-binary-copy/0.log"
Jul 28 18:57:30.060717 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:30.060700    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cl2r6_65b87c8d-03ec-45d8-8d7e-76892c63898c/cni-plugins/0.log"
Jul 28 18:57:30.078193 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:30.078174    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cl2r6_65b87c8d-03ec-45d8-8d7e-76892c63898c/bond-cni-plugin/0.log"
Jul 28 18:57:30.095462 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:30.095441    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cl2r6_65b87c8d-03ec-45d8-8d7e-76892c63898c/routeoverride-cni/0.log"
Jul 28 18:57:30.112529 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:30.112501    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cl2r6_65b87c8d-03ec-45d8-8d7e-76892c63898c/whereabouts-cni-bincopy/0.log"
Jul 28 18:57:30.132083 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:30.132067    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cl2r6_65b87c8d-03ec-45d8-8d7e-76892c63898c/whereabouts-cni/0.log"
Jul 28 18:57:30.174267 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:30.174249    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gt7q9_9e09196f-d3fa-4a41-8d0b-73e65d75979e/kube-multus/0.log"
Jul 28 18:57:30.289406 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:30.289332    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4v2hh_3c98a16c-de7a-4336-b23c-fcb14ec96265/network-metrics-daemon/0.log"
Jul 28 18:57:30.303141 ip-10-0-141-192 kubenswrapper[2583]: I0728 18:57:30.303115    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4v2hh_3c98a16c-de7a-4336-b23c-fcb14ec96265/kube-rbac-proxy/0.log"